fix(nests): close 4 audio-dropout sources across listener + speaker
1. Listener-side pre-roll + bigger playback buffer + audio-priority thread.
- NestPlayer buffers 5 decoded frames (~100 ms) before starting
the AudioPlayer, masking the first-frame underrun that fires
whenever Compose / GC briefly stalls Main.
- AudioTrackPlayer sizes the AudioTrack at max(minBuffer*16, 250 ms)
instead of minBuffer*4 (~80 ms) and writes via a per-instance
audio-priority single-thread executor (Process.THREAD_PRIORITY_AUDIO)
instead of Dispatchers.IO, so WRITE_BLOCKING never contends with
unrelated IO work.
2. MoqLiteSession.subscribe: hoist response typeCode out of the
collect lambda. readVarint advances pos permanently while
readSizePrefixed only rolls back its own length-varint, so a
chunk-split between type and body would re-read the body's size
prefix as the type code on the next chunk and misframe the
response. Mirrors the same fix already in handleInboundBidi.
3. MoqLiteSession.subscribe: register the subscription in the map
BEFORE writing the SUBSCRIBE bytes on the wire. The relay can
open the first group's uni stream before our continuation
re-enters [state] to register; if so, drainOneGroup looked the
id up against an empty map and silently dropped the frame
(~1 frame / 20 ms gap on first attach). Wrap the post-register
writes so a transport-failure unwinds the orphan registration.
4. Hot-swap moq-lite publisher across JWT-refresh boundaries.
- NestMoqLiteBroadcaster: publisher is now @Volatile + supports
swapPublisher(); capture loop snapshots the reference per frame
and resets framesInCurrentGroup on swap.
- MoqLiteNestsSpeaker implements a new internal
HotSwappablePublisherSource interface that exposes
openPublisherForHotSwap(track) without spinning up a broadcaster.
- ReissuingBroadcastHandle keeps a single long-lived broadcaster
across session recycles when the speaker supports hot swap;
legacy IETF / fake speakers fall back to close-then-restart.
- connectReconnectingNestsSpeaker.orchestrator hoists the old-
session close onto a sibling launch so the wrapper can swap
the publisher into the broadcaster on the new session before
the old session's WebTransport drops. Eliminates the previously-
accepted 50–150 ms audible silence at every JWT refresh.
This commit is contained in:
+76
-8
@@ -22,8 +22,12 @@ package com.vitorpamplona.nestsclient.audio
|
||||
|
||||
import android.media.AudioAttributes
|
||||
import android.media.AudioTrack
|
||||
import kotlinx.coroutines.Dispatchers
|
||||
import android.os.Process
|
||||
import kotlinx.coroutines.ExecutorCoroutineDispatcher
|
||||
import kotlinx.coroutines.asCoroutineDispatcher
|
||||
import kotlinx.coroutines.withContext
|
||||
import java.util.concurrent.ExecutorService
|
||||
import java.util.concurrent.Executors
|
||||
import android.media.AudioFormat as AndroidAudioFormat
|
||||
|
||||
/**
|
||||
@@ -45,9 +49,21 @@ import android.media.AudioFormat as AndroidAudioFormat
|
||||
* `MediaRecorder.AudioSource.VOICE_COMMUNICATION` regardless of the
|
||||
* playback usage.
|
||||
*
|
||||
* Buffer sizing: 4× minimum so the producer can fall behind by ~80 ms before
|
||||
* dropouts, which roughly matches the jitter the WebTransport datagram path
|
||||
* introduces over typical mobile networks.
|
||||
* Buffer sizing: target ~250 ms of slack, computed as
|
||||
* `max(minBuffer * 16, 250 ms-equivalent)`. The previous 4× minimum (~80 ms
|
||||
* by the device-reported floor) underran on devices with very small
|
||||
* `getMinBufferSize` returns and on any handset whose decode loop got
|
||||
* stalled by Compose recomposition / GC on Main. 250 ms matches the
|
||||
* jitter-buffer depth Spaces / Clubhouse use for hands-free audio rooms,
|
||||
* and combined with the per-subscription pre-roll in [NestPlayer] keeps
|
||||
* the AudioTrack from underruning across typical mobile network jitter.
|
||||
*
|
||||
* Threading: writes go through a per-instance audio-priority single-thread
|
||||
* dispatcher (`HandlerThread` + [Process.THREAD_PRIORITY_AUDIO]) instead of
|
||||
* `Dispatchers.IO`. The previous shape did one IO dispatcher hop per Opus
|
||||
* frame (~50 hops/sec/speaker) and contended with whatever else `Dispatchers.IO`
|
||||
* was running; an audio-priority dedicated thread gets reliable scheduling
|
||||
* and removes the contention.
|
||||
*/
|
||||
class AudioTrackPlayer(
|
||||
private val usage: Int = AudioAttributes.USAGE_MEDIA,
|
||||
@@ -57,6 +73,18 @@ class AudioTrackPlayer(
|
||||
private var muted: Boolean = false
|
||||
private var volume: Float = 1f
|
||||
|
||||
/**
|
||||
* Dedicated audio-priority single-thread executor for AudioTrack writes.
|
||||
* Lazily created on [start] and shut down on [stop] so a never-started
|
||||
* player doesn't leak a thread. `THREAD_PRIORITY_AUDIO` is the standard
|
||||
* Linux nice level for VoIP / WebRTC playback paths on Android — it sits
|
||||
* above background but below true audio-callback priority, so the OS
|
||||
* scheduler keeps it running through GC / Compose recomposition without
|
||||
* starving other threads.
|
||||
*/
|
||||
private var audioExecutor: ExecutorService? = null
|
||||
private var audioDispatcher: ExecutorCoroutineDispatcher? = null
|
||||
|
||||
override fun start() {
|
||||
if (track != null) return
|
||||
|
||||
@@ -79,7 +107,14 @@ class AudioTrackPlayer(
|
||||
"AudioTrack.getMinBufferSize returned $minBuffer for ${AudioFormat.SAMPLE_RATE_HZ} Hz",
|
||||
)
|
||||
}
|
||||
val bufferBytes = minBuffer * 4
|
||||
// Target ~250 ms of audio: enough headroom so the decode loop can
|
||||
// miss its 20 ms cadence by an order of magnitude before the device
|
||||
// underruns. Take the larger of `minBuffer * 16` and an explicit
|
||||
// 250 ms-equivalent so devices that report a small minBuffer still
|
||||
// get the same wall-clock slack.
|
||||
val targetBytes250Ms =
|
||||
(AudioFormat.SAMPLE_RATE_HZ / 4) * AudioFormat.BYTES_PER_SAMPLE * AudioFormat.CHANNELS
|
||||
val bufferBytes = maxOf(minBuffer * 16, targetBytes250Ms)
|
||||
|
||||
val newTrack =
|
||||
try {
|
||||
@@ -120,14 +155,37 @@ class AudioTrackPlayer(
|
||||
)
|
||||
}
|
||||
applyMuteVolume(newTrack)
|
||||
// Spin up the audio-priority writer thread. The executor is private
|
||||
// to this player instance so per-speaker NestPlayer pumps don't
|
||||
// contend on a shared queue. Priority is set inside the thread's
|
||||
// Runnable because Linux thread priority is per-OS-thread, not
|
||||
// per-Java Thread; `Thread.setPriority` does NOT translate to a
|
||||
// Linux nice level on Android. `Process.setThreadPriority` does.
|
||||
val executor =
|
||||
Executors.newSingleThreadExecutor { r ->
|
||||
Thread(
|
||||
{
|
||||
runCatching { Process.setThreadPriority(Process.THREAD_PRIORITY_AUDIO) }
|
||||
r.run()
|
||||
},
|
||||
"nest-audio-writer",
|
||||
)
|
||||
}
|
||||
audioExecutor = executor
|
||||
audioDispatcher = executor.asCoroutineDispatcher()
|
||||
track = newTrack
|
||||
}
|
||||
|
||||
override suspend fun enqueue(pcm: ShortArray) {
|
||||
val t = track ?: throw AudioException(AudioException.Kind.PlaybackFailed, "player not started")
|
||||
// AudioTrack.write blocks if the internal buffer is full. Run on IO so
|
||||
// we don't stall a coroutine dispatcher backed by a small thread pool.
|
||||
withContext(Dispatchers.IO) {
|
||||
val dispatcher =
|
||||
audioDispatcher
|
||||
?: throw AudioException(AudioException.Kind.PlaybackFailed, "audio dispatcher not initialized")
|
||||
// AudioTrack.write blocks if the internal buffer is full. Run on the
|
||||
// per-instance audio-priority writer thread so the WRITE_BLOCKING
|
||||
// suspension is on a thread the OS schedules tightly, and so we
|
||||
// don't compete with whatever else `Dispatchers.IO` is running.
|
||||
withContext(dispatcher) {
|
||||
val written = t.write(pcm, 0, pcm.size, AudioTrack.WRITE_BLOCKING)
|
||||
if (written < 0) {
|
||||
throw AudioException(
|
||||
@@ -155,6 +213,16 @@ class AudioTrackPlayer(
|
||||
runCatching { t.flush() }
|
||||
runCatching { t.stop() }
|
||||
runCatching { t.release() }
|
||||
// Tear down the audio-priority writer. close() on the
|
||||
// ExecutorCoroutineDispatcher shuts down the executor (and the
|
||||
// underlying single thread) — pending writes are abandoned rather
|
||||
// than blocked on, since the AudioTrack itself has already been
|
||||
// stopped + released two lines up so any further `write` would
|
||||
// throw IllegalStateException anyway.
|
||||
audioDispatcher?.close()
|
||||
audioDispatcher = null
|
||||
audioExecutor?.let { runCatching { it.shutdownNow() } }
|
||||
audioExecutor = null
|
||||
}
|
||||
|
||||
private fun applyMuteVolume(track: AudioTrack) {
|
||||
|
||||
Reference in New Issue
Block a user