fix(nestsClient): fix race + leaks + Android encoder/decoder bugs from second audit
A fresh audit caught a real race in the round-1 subscribe() rewrite plus several distinct bugs in the Android MediaCodec layer. **MoqLiteSession.subscribe() race regression** — the launched collector that reads the SubscribeResponse and watches for peer FIN ran cleanup (remove from subscriptionsBySubscribeId, close frames) before subscribe() itself reached the post-await registration. If the publisher FIN'd immediately after sending Ok, the collector exited first against an empty map; subscribe() then registered the subscription, leaving frames in the map with no live collector to ever close it on transport drop. Consumer hung forever — exactly the failure mode round-1 fix #3 was supposed to prevent. Fix: pre-register the subscription BEFORE launching the collector. Drop unused `ok` field from ListenerSubscription. **MoqLiteNestsSpeaker.startBroadcasting publisher leak** — if session.publish() succeeded but broadcaster.start() then threw (mic permission denied, AudioRecord allocation failure), the publisher was never closed and stayed registered as session.activePublisher, permanently blocking subsequent startBroadcasting calls. **runCatching{suspend close} swallowing CancellationException** — MoqLiteNestsSpeaker.close, MoqLiteBroadcastHandle.close, MoqLiteNestsListener.close all wrapped suspending closes in runCatching, breaking structured cancellation when the parent scope cancelled teardown. Replaced with explicit cancel-rethrowing try/catch. **MediaCodecOpusDecoder ArrayList<Short> boxing on every audio frame** — at 50 fps × 960 samples × N speakers ≈ 48 000 boxed Short allocations/sec/speaker on the audio hot path. Rewritten to write directly into a pre-sized ShortArray via ShortBuffer.get(dst, off, len). **MediaCodecOpusEncoder bugs** - KEY_AAC_PROFILE = AACObjectLC was being set on an audio/opus encoder. Meaningless for Opus; stricter Codec2 stacks on Android 13+ reject the configure() call with IllegalArgumentException and surface as DeviceUnavailable. Removed. - The drain loop's INFO_OUTPUT_FORMAT_CHANGED branch had no progress guard. A buggy encoder re-emitting FORMAT_CHANGED without producing output would busy-spin against the 10 ms dequeue timeout. Now absorbed at most once per encode call. Same guard added to the decoder. Adds regression test: - frames_flow_completes_when_peer_FINs_immediately_after_Ok 224 tests pass, 0 failures. Android target compiles clean.
This commit is contained in:
Claude
+25
-11
@@ -74,33 +74,47 @@ class MediaCodecOpusDecoder : OpusDecoder {
|
||||
// Drain whatever output is ready right now. A single Opus packet
|
||||
// typically yields exactly one output buffer, but on some devices the
|
||||
// first call returns INFO_OUTPUT_FORMAT_CHANGED before the PCM frame.
|
||||
val collected = ArrayList<Short>(AudioFormat.FRAME_SIZE_SAMPLES)
|
||||
while (true) {
|
||||
//
|
||||
// Write the decoded samples directly into a pre-sized ShortArray
|
||||
// via ShortBuffer.get(dst, off, len) — the previous shape went via
|
||||
// ArrayList<Short>, which boxed every PCM sample (one heap object
|
||||
// per Short × 960 samples × 50 fps × N speakers ≈ 48 000
|
||||
// allocations/sec/speaker on the audio hot path).
|
||||
val out = ShortArray(AudioFormat.FRAME_SIZE_SAMPLES)
|
||||
var outPos = 0
|
||||
var formatChangeAbsorbed = false
|
||||
drain@ while (true) {
|
||||
val outputIndex = codec.dequeueOutputBuffer(bufferInfo, DEQUEUE_TIMEOUT_US)
|
||||
when {
|
||||
outputIndex >= 0 -> {
|
||||
val outputBuffer =
|
||||
codec.getOutputBuffer(outputIndex)
|
||||
?: continue
|
||||
?: continue@drain
|
||||
if (bufferInfo.size > 0) {
|
||||
outputBuffer.position(bufferInfo.offset)
|
||||
outputBuffer.limit(bufferInfo.offset + bufferInfo.size)
|
||||
val shorts = outputBuffer.order(ByteOrder.nativeOrder()).asShortBuffer()
|
||||
val tmp = ShortArray(shorts.remaining())
|
||||
shorts.get(tmp)
|
||||
for (s in tmp) collected.add(s)
|
||||
val toCopy = minOf(shorts.remaining(), out.size - outPos)
|
||||
if (toCopy > 0) {
|
||||
shorts.get(out, outPos, toCopy)
|
||||
outPos += toCopy
|
||||
}
|
||||
}
|
||||
codec.releaseOutputBuffer(outputIndex, false)
|
||||
if (bufferInfo.flags and MediaCodec.BUFFER_FLAG_END_OF_STREAM != 0) break
|
||||
// No more buffered output for this packet.
|
||||
if (bufferInfo.size == 0) break
|
||||
if (collected.size >= AudioFormat.FRAME_SIZE_SAMPLES) break
|
||||
if (outPos >= out.size) break
|
||||
}
|
||||
|
||||
outputIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED -> {
|
||||
// The format change carries no audio data; loop to read
|
||||
// the actual PCM frame.
|
||||
continue
|
||||
// The format change carries no audio data; absorb it
|
||||
// once and loop to read the actual PCM frame. Buggy
|
||||
// decoders that re-fire format-change without
|
||||
// producing output would otherwise busy-spin.
|
||||
if (formatChangeAbsorbed) break
|
||||
formatChangeAbsorbed = true
|
||||
continue@drain
|
||||
}
|
||||
|
||||
outputIndex == MediaCodec.INFO_TRY_AGAIN_LATER -> {
|
||||
@@ -112,7 +126,7 @@ class MediaCodecOpusDecoder : OpusDecoder {
|
||||
}
|
||||
}
|
||||
}
|
||||
return ShortArray(collected.size) { collected[it] }
|
||||
return if (outPos == out.size) out else out.copyOf(outPos)
|
||||
}
|
||||
|
||||
override fun release() {
|
||||
|
||||
+13
-8
@@ -21,7 +21,6 @@
|
||||
package com.vitorpamplona.nestsclient.audio
|
||||
|
||||
import android.media.MediaCodec
|
||||
import android.media.MediaCodecInfo
|
||||
import android.media.MediaFormat
|
||||
import java.nio.ByteOrder
|
||||
|
||||
@@ -76,7 +75,12 @@ class MediaCodecOpusEncoder(
|
||||
presentationTimeUs += FRAME_DURATION_US
|
||||
|
||||
// One PCM frame produces one Opus packet (sometimes after one warmup
|
||||
// round). Drain the output queue once.
|
||||
// round). Drain the output queue once. The format-change signal
|
||||
// fires at most once on encoder startup; absorb it then re-poll,
|
||||
// but never loop on it (some buggy encoders re-emit FORMAT_CHANGED
|
||||
// without producing output, which would otherwise busy-spin at
|
||||
// 100 Hz against the 10 ms dequeue timeout).
|
||||
var formatChangeAbsorbed = false
|
||||
while (true) {
|
||||
val outputIndex = codec.dequeueOutputBuffer(bufferInfo, DEQUEUE_TIMEOUT_US)
|
||||
when {
|
||||
@@ -91,6 +95,8 @@ class MediaCodecOpusEncoder(
|
||||
}
|
||||
|
||||
outputIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED -> {
|
||||
if (formatChangeAbsorbed) return ByteArray(0)
|
||||
formatChangeAbsorbed = true
|
||||
continue
|
||||
}
|
||||
|
||||
@@ -128,12 +134,11 @@ class MediaCodecOpusEncoder(
|
||||
).apply {
|
||||
setInteger(MediaFormat.KEY_BIT_RATE, bitrate)
|
||||
setInteger(MediaFormat.KEY_PCM_ENCODING, android.media.AudioFormat.ENCODING_PCM_16BIT)
|
||||
// Encoder-side AAC/Opus profile selection: SignalingDelaySamples
|
||||
// is implicit; nothing else required for mono speech.
|
||||
setInteger(
|
||||
MediaFormat.KEY_AAC_PROFILE,
|
||||
MediaCodecInfo.CodecProfileLevel.AACObjectLC,
|
||||
)
|
||||
// KEY_AAC_PROFILE on an audio/opus encoder is meaningless
|
||||
// (Opus has no AAC profile); historically it was silently
|
||||
// ignored, but stricter Codec2 stacks on Android 13+
|
||||
// reject the configure() call with IllegalArgumentException
|
||||
// and surface as DeviceUnavailable.
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user