fix(nests): close receiver-audio dropout cluster + auto-start broadcast on host create

Six changes that together close out the bug class the user reported
(receiver "blinks green ring for ~5 s then stops") and the related
host-side "circular spinner until first unmute" UX issue. Each is
defensible independently; together they form a defense-in-depth
against the moq-rs production-relay forward-queue cliff documented
in `nestsClient/plans/2026-05-01-quic-stream-cliff-investigation.md`.

#3 — `NestMoqLiteBroadcaster.onLevel` only fires when
`current.send(opus)` returned `true` (frame reached an inbound
subscriber). Two-phone logs showed the host's green/glow firing for
the first ~7 s while no listener was attached and after a relay-side
cliff while no audio was on the wire. The local "I'm broadcasting"
ring now tracks the wire, not the runCatching outcome.

#5 — `DEFAULT_FRAMES_PER_GROUP: 5 → 10` (200 ms of audio per
group, 5 streams/sec). Two-phone production logs showed the relay
still cliffing at ~135 streams under sustained load with the old
5-frame default — same bug class the plan thought it had closed,
just shifted by half. Doubling the pack roughly doubles the
worst-case cliff window. Kept the existing `framesPerGroup = 5`
test scenarios intact since they're explicit on the call site.

#4 — `SUBSCRIBE_RETRY_BACKOFF_INITIAL_MS: 250 → 100`. Logs of the
join path showed 5 retries (250+500+1000+1000+1000 = 3.75 s) of
"subscribe stream FIN before reply" before the relay observed the
broadcast's announce. New ladder is 100+200+400+800+1000 = 2.5 s.
The MAX cap stays at 1000 ms so a permanently-gone publisher still
doesn't hammer the relay.

#2 — `NestViewModel.openSubscription` now passes
`maxLatencyMs = 500L` on every speaker SUBSCRIBE (new constant
`ROOM_AUDIO_MAX_LATENCY_MS`). Tells moq-rs to evict groups older
than 500 ms from the per-subscriber forward queue rather than
queuing them up to the relay's `MAX_GROUP_AGE = 30 s` default —
caps the queue growth that triggers the cliff. The wire support
existed in the listener; only the call site was passing 0L.

#1 — listener-side cliff detector. New `cliffDetectorJob` in
`NestViewModel` periodically scans `activeSubscriptions` against
`lastFrameAt` (per-pubkey monotonic TimeMark, updated in
`onSpeakerActivity`). When a speaker has been announced Active AND
we have an active subscription AND no frames have arrived for
`ROOM_AUDIO_CLIFF_TIMEOUT_MS = 4 s`, force a `recycleSession()` so
the orchestrator opens a fresh QUIC transport — a clean reset of
the relay's per-subscriber forward queue. `ROOM_AUDIO_CLIFF_COOLDOWN_MS = 8 s`
prevents back-to-back recycles while the new session is mid-handshake.
Started from `launchConnect` after `observeAnnounces`, cancelled in
`teardown`. Per-pubkey `lastFrameAt` entry dropped on
`closeSubscription` so a removed speaker doesn't trigger a stale
recycle.

#6 — auto-start broadcast pre-muted on host create.
`NestViewModel.startBroadcast` gets an `initialMuted: Boolean = false`
parameter; when `true` the broadcaster opens the publisher session,
calls `setMuted(true)` before the UI flips to `Broadcasting`, and the
mic stays open + capture loop keeps running with `if (muted) continue`
gating sends. New `LaunchedEffect(speakerPubkeyHex)` in
`OnStageIdleControls` calls `startBroadcast(initialMuted = true)`
automatically when mic permission is already granted, so a host who
just created a room sees their avatar transition to "Live (silent)"
and listeners can subscribe immediately without waiting for the host
to tap "Talk" first. Tap-to-talk path also passes `initialMuted = true`
now — unmute is a separate, explicit step on the mic toggle that
appears once we're in `Broadcasting(isMuted = true)`. Permission-
denied case still requires the manual Talk-button tap to launch the
runtime permission prompt.

Diagnostic logs from the previous two debug commits remain in place
(throttled to ≤ 1 line/sec/speaker at 50 fps capture cadence) so the
next two-phone repro can validate the fixes against logcat directly.
This commit is contained in:
Claude
2026-05-05 16:44:39 +00:00
parent be8dd0a3bc
commit cb61082bc4
4 changed files with 276 additions and 30 deletions
@@ -50,6 +50,7 @@ import androidx.compose.material3.OutlinedButton
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.saveable.rememberSaveable
@@ -270,8 +271,11 @@ private fun OnStageControls(
is BroadcastUiState.Failed -> {
// Reason is shown in the status strip; this button retries.
// Same auto-muted semantics as the idle path: the retry
// should bring the publisher back up without opening the
// mic, then the user can unmute deliberately.
TalkButton(
onClick = { viewModel.startBroadcast(speakerPubkeyHex) },
onClick = { viewModel.startBroadcast(speakerPubkeyHex, initialMuted = true) },
contentDescription = stringRes(R.string.nest_talk),
)
LeaveStageButton(onClick = { viewModel.setOnStage(false) })
@@ -289,17 +293,43 @@ private fun OnStageIdleControls(
val permissionLauncher =
rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission()) { granted ->
permissionDenied = !granted
if (granted) viewModel.startBroadcast(speakerPubkeyHex)
// Permission grant on a Talk-button tap should start a
// *muted* broadcast — auto-start semantics. The host
// taps Talk to claim the on-stage slot; unmute is a
// separate, deliberate action below.
if (granted) viewModel.startBroadcast(speakerPubkeyHex, initialMuted = true)
}
// The launcher callback never fires for Settings-deep-link grants, so
// re-check on every recomposition to auto-clear the warning.
val showDenialWarning =
permissionDenied && !context.hasMicPermission()
// Auto-start the muted broadcast as soon as the host (or any
// on-stage speaker) lands on this composable WITH mic permission
// already granted. Without this, a host who creates a room sees
// a "Connecting..." spinner over their avatar until they tap Talk
// — the publisher session only opens on tap, so listeners can't
// subscribe yet, and the `connectingSpeakers` overlay sticks. By
// pre-opening the publisher in muted state here, listeners get
// an immediate Active announce and the host's avatar leaves the
// spinner state on its own. Unmute is still a separate tap on
// the mic toggle that appears once we transition to
// `BroadcastUiState.Broadcasting(isMuted = true)`.
//
// Permission-denied case is intentionally left to the manual
// Talk-button tap below: we don't want a host to be auto-prompted
// for the mic the instant they enter the room. Tapping Talk is
// the consent gesture that triggers `permissionLauncher`.
LaunchedEffect(speakerPubkeyHex) {
if (context.hasMicPermission()) {
viewModel.startBroadcast(speakerPubkeyHex, initialMuted = true)
}
}
TalkButton(
onClick = {
if (context.hasMicPermission()) {
viewModel.startBroadcast(speakerPubkeyHex)
viewModel.startBroadcast(speakerPubkeyHex, initialMuted = true)
} else {
permissionLauncher.launch(Manifest.permission.RECORD_AUDIO)
}
@@ -265,6 +265,41 @@ class NestViewModel(
private val activeSubscriptions = mutableMapOf<String, ActiveSubscription>()
private val speakingExpiryJobs = mutableMapOf<String, Job>()
/**
* Monotonic [kotlin.time.TimeMark] of the last MoQ object delivered
* to the consumer flow per speaker pubkey. Updated in
* [onSpeakerActivity] alongside the speaking-now flag. Read by
* [cliffDetectorJob] to spot the relay-side forward-queue cliff
* documented in
* `nestsClient/plans/2026-05-01-quic-stream-cliff-investigation.md`:
* the relay stops opening new uni streams to the listener while
* the announce stream still says the broadcast is Active and the
* subscribe bidi is still open from QUIC's POV — meaning no other
* layer notices.
*/
private val lastFrameAt = mutableMapOf<String, kotlin.time.TimeSource.Monotonic.ValueTimeMark>()
/**
* Periodic scan of [activeSubscriptions] vs [_announcedSpeakers]
* vs [lastFrameAt]. When a speaker has been announced Active
* AND we have an active subscription AND no frames have arrived
* for [ROOM_AUDIO_CLIFF_TIMEOUT_MS], force a `recycleSession()`
* so the orchestrator opens a fresh QUIC transport — a clean
* recovery from the relay-side per-subscriber forward-queue
* starvation that has no QUIC- or moq-lite-level signal.
*/
private var cliffDetectorJob: Job? = null
/**
* [kotlin.time.TimeMark] of the most recent recycle the cliff
* detector triggered, or `null` if it has never fired. Acts as a
* cooldown so a single cliff event doesn't kick off multiple
* recycles back-to-back while the wrapper is mid-handshake on
* the new session — the new session has no incoming frames yet,
* which would otherwise trip the detector again immediately.
*/
private var lastCliffRecycleAt: kotlin.time.TimeSource.Monotonic.ValueTimeMark? = null
/**
* Per-speaker catalog-fetch coroutines. Each entry is the
* background `subscribeCatalog` collector launched in
@@ -561,7 +596,10 @@ class NestViewModel(
* session over a separate WebTransport — nests' protocol uses one
* session per direction.
*/
fun startBroadcast(speakerPubkeyHex: String) {
fun startBroadcast(
speakerPubkeyHex: String,
initialMuted: Boolean = false,
) {
if (closed || !canBroadcast) return
if (_uiState.value.connection !is ConnectionUiState.Connected) return
if (_uiState.value.broadcast is BroadcastUiState.Broadcasting ||
@@ -608,7 +646,26 @@ class NestViewModel(
},
)
broadcastHandle = handle
_uiState.update { it.copy(broadcast = BroadcastUiState.Broadcasting(isMuted = false)) }
// Auto-start path (room creation): the host wants
// listeners to be able to subscribe BEFORE they hit
// unmute, so we open the publisher session pre-muted.
// The mic stays open + the broadcaster keeps
// capturing + encoding, but `setMuted(true)` makes
// every send drop on the floor inside
// `NestMoqLiteBroadcaster.start`'s `if (muted) continue`
// gate. Listener-side announces fire as soon as the
// publisher registers with the relay, so the audience
// can subscribe immediately and switch from
// "Connecting…" to "Live (silent)" while they wait
// for the host's first word. Composes with `focusMuted`
// exactly the same way [setMicMuted] does so an
// inbound phone call during the auto-start window
// doesn't get a silent unmute.
if (initialMuted) {
val effective = initialMuted || focusMuted
runCatching { handle.setMuted(effective) }
}
_uiState.update { it.copy(broadcast = BroadcastUiState.Broadcasting(isMuted = initialMuted)) }
} catch (ce: CancellationException) {
throw ce
} catch (t: Throwable) {
@@ -855,6 +912,7 @@ class NestViewModel(
listener = l
observeListenerState(l)
observeAnnounces(l)
startCliffDetector()
} catch (ce: CancellationException) {
throw ce
} catch (t: Throwable) {
@@ -915,6 +973,11 @@ class NestViewModel(
if (rawAudioLevels.remove(slot.pubkey) != null) {
_audioLevels.value = rawAudioLevels.toMap()
}
// Drop the cliff-detector heartbeat for this pubkey so the next
// tick doesn't see a stale "last frame was 30s ago" entry for a
// speaker we already stopped subscribing to and trigger a
// gratuitous recycle.
lastFrameAt.remove(slot.pubkey)
if (roomPlayer != null || handle != null) {
viewModelScope.launch {
roomPlayer?.runCatching { stop() }
@@ -961,7 +1024,21 @@ class NestViewModel(
) {
if (closed || activeSubscriptions[pubkey] !== slot) return
try {
val handle = l.subscribeSpeaker(pubkey)
// [maxLatencyMs] tells the relay our staleness tolerance:
// groups older than this should be evicted, not queued.
// With `0L` (the wire default the JS reference uses) the
// relay's per-subscriber forward queue grows unbounded
// until it cliffs — see
// `nestsClient/plans/2026-05-01-quic-stream-cliff-investigation.md`.
// 500 ms is long enough to ride out a typical relay-side
// hiccup without pruning a healthy group, short enough
// that a wedged subscriber drains its buffer before the
// queue overflows. Combined with the cliff-detector below
// that triggers `recycleSession()` when frames stop
// arriving despite the broadcast still being announced,
// this is a defense-in-depth pair against the moq-rs
// forward-queue starvation behaviour.
val handle = l.subscribeSpeaker(pubkey, maxLatencyMs = ROOM_AUDIO_MAX_LATENCY_MS)
// Re-check after the suspending subscribeSpeaker — the user
// may have removed this speaker via updateSpeakers / disconnected
// while the SUBSCRIBE was in flight. If so, abandon the handle
@@ -1089,6 +1166,10 @@ class NestViewModel(
announcesJob = null
levelEmitterJob?.cancel()
levelEmitterJob = null
cliffDetectorJob?.cancel()
cliffDetectorJob = null
lastFrameAt.clear()
lastCliffRecycleAt = null
// Audio-focus observation only ends on the final teardown —
// a transient disconnect+reconnect (user retry, room swap)
// keeps the bus subscription alive so a focus loss that
@@ -1175,6 +1256,14 @@ class NestViewModel(
*/
private fun onSpeakerActivity(pubkey: String) {
if (closed) return
// Cliff-detector heartbeat: each MoQ object proves the
// listener-side relay-forward queue is still flowing for this
// speaker. The detector only acts when an announced speaker's
// last-frame timestamp ages past `ROOM_AUDIO_CLIFF_TIMEOUT_MS`,
// so an active stream resets it on every frame.
lastFrameAt[pubkey] =
kotlin.time.TimeSource.Monotonic
.markNow()
speakingExpiryJobs[pubkey]?.cancel()
// First frame for this subscription — clear the buffering
// overlay. Subsequent frames are no-ops here.
@@ -1191,6 +1280,61 @@ class NestViewModel(
}
}
/**
* Periodic scan that detects the relay-side forward-queue cliff:
* an announced speaker whose subscription is open but hasn't
* delivered any object in [ROOM_AUDIO_CLIFF_TIMEOUT_MS]. Triggers
* `recycleSession()` on the listener so the wrapper opens a fresh
* QUIC transport — the new session has a fresh per-subscriber
* queue on the relay's side.
*
* Idempotent on start. Stops itself when the listener is gone
* (explicit teardown signal) — no-ops on transient empty
* `activeSubscriptions` so an audience-only listener doesn't
* needlessly recycle.
*/
private fun startCliffDetector() {
if (cliffDetectorJob?.isActive == true) return
cliffDetectorJob =
viewModelScope.launch {
while (true) {
delay(ROOM_AUDIO_CLIFF_CHECK_INTERVAL_MS)
if (closed) return@launch
val l = listener ?: continue
if (_uiState.value.connection !is ConnectionUiState.Connected) continue
// Cooldown: if we very recently triggered a recycle,
// give the orchestrator time to bring up the new
// session before re-checking. A new session has
// nothing in `lastFrameAt` yet which would otherwise
// immediately re-trigger.
val sinceRecycle = lastCliffRecycleAt?.elapsedNow()
if (sinceRecycle != null &&
sinceRecycle.inWholeMilliseconds < ROOM_AUDIO_CLIFF_COOLDOWN_MS
) {
continue
}
val announced = _announcedSpeakers.value
val stalled =
activeSubscriptions
.asSequence()
.filter { (pubkey, slot) -> slot.isPlaying && pubkey in announced }
.filter { (pubkey, _) ->
val mark = lastFrameAt[pubkey] ?: return@filter false
mark.elapsedNow().inWholeMilliseconds >= ROOM_AUDIO_CLIFF_TIMEOUT_MS
}.map { it.key }
.toList()
if (stalled.isEmpty()) continue
com.vitorpamplona.quartz.utils.Log.w("NestRx") {
"cliff-detector: announced+subscribed but silent for ≥${ROOM_AUDIO_CLIFF_TIMEOUT_MS}ms — recycling session. stalled=$stalled"
}
lastCliffRecycleAt =
kotlin.time.TimeSource.Monotonic
.markNow()
runCatching { l.recycleSession() }
}
}
}
private fun clearSpeaking(pubkey: String) {
speakingExpiryJobs.remove(pubkey)
if (_uiState.value.speakingNow.contains(pubkey)) {
@@ -1442,6 +1586,54 @@ const val ROOM_PLAYER_PREROLL_FRAMES: Int = 5
*/
const val LEVEL_TICK_MS: Long = 100L
/**
* `max_latency` we send on every speaker SUBSCRIBE. Tells the relay
* to evict groups older than this from the per-subscriber forward
* queue rather than holding them up to the relay's `MAX_GROUP_AGE`
* default (30 s). Defends against the moq-rs forward-queue cliff
* documented in
* `nestsClient/plans/2026-05-01-quic-stream-cliff-investigation.md`.
* 500 ms = 25 Opus frames at 20 ms cadence — long enough to ride
* out a typical hiccup without dropping a healthy frame, short
* enough to keep the relay's per-subscriber queue bounded.
*/
const val ROOM_AUDIO_MAX_LATENCY_MS: Long = 500L
/**
* Inactivity threshold for the listener-side cliff detector. If
* a speaker is `_announcedSpeakers` (= relay says they're broadcasting)
* AND the wrapper's MoqObject flow doesn't deliver a frame for this
* many milliseconds, the listener forces a `recycleSession()` so the
* orchestrator opens a fresh QUIC transport. Resets the relay's
* per-subscriber forward queue.
*
* 4 s gives enough headroom to ride out a publisher-side group-rollover
* hiccup (typically < 200 ms) without false-positive recycling, while
* keeping the audible gap from a real cliff to ~5 s of silence at
* worst (4 s detection + ~1 s reconnect handshake).
*/
const val ROOM_AUDIO_CLIFF_TIMEOUT_MS: Long = 4_000L
/**
* How often the cliff detector wakes up to scan
* `activeSubscriptions` against `lastFrameAt`. 1 s is a coarse
* enough cadence not to cost anything (one map walk per second)
* while keeping detection latency well under the 4 s threshold.
*/
const val ROOM_AUDIO_CLIFF_CHECK_INTERVAL_MS: Long = 1_000L
/**
* Cooldown after a cliff-detector-driven recycle. Suppresses
* back-to-back recycles while the wrapper is mid-handshake on the
* fresh session — a brand-new session has no `lastFrameAt` entries
* yet, which would otherwise immediately re-trigger the detector
* on the next 1 s tick. Long enough to cover the typical reconnect
* handshake plus a couple of audio-frame round trips, short enough
* that a recycle that didn't actually clear the cliff is recovered
* from quickly.
*/
const val ROOM_AUDIO_CLIFF_COOLDOWN_MS: Long = 8_000L
/**
* How long a kind-7 reaction stays in
* [NestViewModel.recentReactions] before the eviction sweep
@@ -476,18 +476,23 @@ private class ReconnectingHandle(
// before the publisher exists, and the relay FINs the bidi
// without a SubscribeOk/Drop reply.
//
// - INITIAL = 250 ms: under moq-rs's typical announce-propagation
// latency (< 200 ms in production traces), so the very first
// retry usually succeeds and the listener-side buffer hides
// the gap.
// - Doubles each miss → 500 ms → 1 000 ms.
// - MAX = 1 000 ms: matches the previous flat constant; long
// enough that a never-arriving publisher doesn't hammer the
// relay, short enough to stay under the wrapper SharedFlow's
// ~1.3 s buffer once playback is rolling on the next attempt.
// - INITIAL = 100 ms: lower than the prior 250 ms because
// join-time logs (`claude/fix-nests-audio-receiver-HCgOY`)
// showed the listener was paying ~4 s of dead air on every
// first-join while the broadcaster's session warmed up
// on the relay (5 retries: 250+500+1000+1000+1000 = 3.75 s).
// 100 ms initial halves that to ~1.9 s
// (100+200+400+800+1000) without thrashing the relay on a
// permanently-gone publisher (the MAX cap below still
// bounds the total per-attempt rate).
// - Doubles each miss → 200 ms → 400 ms → 800 ms → 1 000 ms.
// - MAX = 1 000 ms: long enough that a never-arriving
// publisher doesn't hammer the relay; short enough to stay
// under the wrapper SharedFlow's ~1.3 s buffer once playback
// is rolling on the next attempt.
// - Reset on first successful subscribe so a subsequent
// publisher-cycle gap doesn't inherit the previous saturation.
private const val SUBSCRIBE_RETRY_BACKOFF_INITIAL_MS = 250L
private const val SUBSCRIBE_RETRY_BACKOFF_INITIAL_MS = 100L
private const val SUBSCRIBE_RETRY_BACKOFF_MAX_MS = 1_000L
private val SYNTH_OK =
@@ -251,6 +251,14 @@ class NestMoqLiteBroadcaster(
"broadcaster sent frame #$sentFrames (group $framesInCurrentGroup/$framesPerGroup)"
}
}
// Only tap the speaking-ring on a frame
// that actually reached an inbound
// subscriber. Without this gate the local
// ring lights up during the pre-subscribe
// window AND after a relay-side cliff —
// the speaker would believe they're being
// heard when no audio is on the wire.
onLevel(peakAmplitude(pcm))
} else {
droppedNoSubFrames += 1
if (droppedNoSubFrames % SEND_LOG_THROTTLE == 0L) {
@@ -259,12 +267,6 @@ class NestMoqLiteBroadcaster(
}
}
}
// Fire the speaking-ring tap only on
// a successful send. If the publisher
// throws (transport gone, peer dead),
// the frame didn't actually go out and
// the UI shouldn't claim we're talking.
onLevel(peakAmplitude(pcm))
}.onFailure { t ->
if (t is CancellationException) throw t
consecutiveSendErrors += 1
@@ -382,15 +384,32 @@ class NestMoqLiteBroadcaster(
companion object {
/**
* Default moq-lite group size = 5 Opus frames ≈ 100 ms of audio.
* Picked to keep the QUIC uni-stream creation rate
* (10 streams/sec at 20 ms cadence) under the production
* nostrnests relay's sustained per-subscriber forward
* ceiling (~40 streams/sec) while still giving late-joining
* subscribers a sub-100 ms initial audio gap. See
* [framesPerGroup] kdoc for the full rationale + history.
* Default moq-lite group size = 10 Opus frames ≈ 200 ms of audio.
*
* Picked to keep the QUIC uni-stream creation rate (5 streams/sec
* at 20 ms cadence) well under the production nostrnests relay's
* per-subscriber forward queue cliff. Two-phone production tests
* (`claude/fix-nests-audio-receiver-HCgOY` logcat) showed the
* relay still cliffed at ~135 streams under sustained load even
* with the old 5-frame default (10 streams/sec), giving ~13 s of
* audio before the listener's incoming uni-stream flow stopped
* — the same bug class documented in
* `nestsClient/plans/2026-05-01-quic-stream-cliff-investigation.md`,
* just shifted by half. Doubling the pack to 10 cuts the rate
* in half again and roughly doubles the cliff window in the
* worst case while halving the relay-side stream-handling
* pressure that triggers it.
*
* Belt-and-suspenders: pair this with the listener-side
* cliff-detector in `NestViewModel` (no-data-while-announced
* triggers `recycleSession()`) so the room recovers from a
* cliff event regardless of the exact threshold.
*
* Late-join gap: ≤ 200 ms (one group boundary) instead of
* ≤ 100 ms with `framesPerGroup = 5` — still imperceptible
* for live audio rooms.
*/
const val DEFAULT_FRAMES_PER_GROUP: Int = 5
const val DEFAULT_FRAMES_PER_GROUP: Int = 10
/**
* Maximum consecutive [MoqLitePublisherHandle.send] / [endGroup]