From 96cfa1235aeb74dfb68e24118cb45f8fdbae956b Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Wed, 6 May 2026 17:37:38 +0000
Subject: [PATCH] fix(nests): T8 skip BUFFER_FLAG_CODEC_CONFIG outputs in
 MediaCodecOpusEncoder
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Android's `audio/opus` MediaCodec encoder emits `BUFFER_FLAG_CODEC_CONFIG`
output buffers BEFORE any audio buffer — typically the 19-byte OpusHead
identification header per RFC 7845, and (on some Codec2 stacks) the
OpusTags comment header. These are decoder-config blobs, NOT audio
frames. We weren't filtering them, so the first 1–2 wire frames every
encoder lifetime were OpusHead bytes wrapped in our legacy-container
varint(timestamp_us) prefix.

The web watcher's WebCodecs `AudioDecoder` handles this by burning
warmup slots (`@moq/watch/src/audio/decoder.ts`'s `warmed <= 3` policy)
— so the listener mostly recovers — but on Android Codec2 stacks that
emit BOTH OpusHead AND OpusTags as separate CSD buffers, two of the
three warmup slots get absorbed by metadata and the listener hears a
tiny click on the next group rollover. The hot-swap path
(`ReconnectingNestsSpeaker`) repeats the warmup on every JWT refresh,
so this fired once every 9 minutes in production.

Filter via `bufferInfo.flags and MediaCodec.BUFFER_FLAG_CODEC_CONFIG`
inside `encode`'s output-drain loop. CSD buffers are released and the
loop continues to the next dequeue rather than returning the bytes.
Logged once per encoder lifetime via `loggedCsdSkip` so we have proof
the path fired without flooding logcat — a stack that emits CSD on
every frame would otherwise be noisy.

Returning `ByteArray(0)` for the first call (because the only output
was a CSD + format-change pair) is unchanged behaviour and the
broadcaster's `if (opus.isEmpty()) continue` already handles it.

https://claude.ai/code/session_014JfZJHSTvyYYWJbC9VbB47
---
 .../audio/MediaCodecOpusEncoder.kt            | 38 +++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/nestsClient/src/androidMain/kotlin/com/vitorpamplona/nestsclient/audio/MediaCodecOpusEncoder.kt b/nestsClient/src/androidMain/kotlin/com/vitorpamplona/nestsclient/audio/MediaCodecOpusEncoder.kt
index 5de9f3911..fbdec2790 100644
--- a/nestsClient/src/androidMain/kotlin/com/vitorpamplona/nestsclient/audio/MediaCodecOpusEncoder.kt
+++ b/nestsClient/src/androidMain/kotlin/com/vitorpamplona/nestsclient/audio/MediaCodecOpusEncoder.kt
@@ -56,6 +56,16 @@ class MediaCodecOpusEncoder(
     private var presentationTimeUs: Long = 0L
     private var released = false
 
+    /**
+     * Latch so a single conspicuous log fires the FIRST time we drop a
+     * [MediaCodec.BUFFER_FLAG_CODEC_CONFIG] output buffer. Without
+     * this latch the same encoder can produce 2-3 CSD buffers in a row
+     * (OpusHead + OpusTags on some Android Codec2 stacks) and we'd
+     * spam the log; with it, we record the fact once per encoder
+     * lifetime and stay quiet thereafter.
+     */
+    private var loggedCsdSkip = false
+
     override fun encode(pcm: ShortArray): ByteArray {
         check(!released) { "encoder released" }
         require(pcm.isNotEmpty()) { "PCM frame must not be empty" }
@@ -86,6 +96,34 @@ class MediaCodecOpusEncoder(
             when {
                 outputIndex >= 0 -> {
                     val outputBuffer = codec.getOutputBuffer(outputIndex) ?: continue
+                    // CODEC_CONFIG buffers carry codec-specific data
+                    // (CSD): on the Android `audio/opus` encoder these
+                    // are the 19-byte OpusHead identification header
+                    // and (on some Codec2 stacks) the OpusTags
+                    // comment header. They are NOT Opus packets — they
+                    // are decoder-config blobs that should be supplied
+                    // out-of-band on the receive side (we already do
+                    // this in `MediaCodecOpusDecoder.buildOpusIdHeader`).
+                    // Sending them to the wire as audio frames means
+                    // the watcher's WebCodecs `AudioDecoder.decode`
+                    // sees garbage in the first frame, burning a
+                    // warmup slot and producing a click on group
+                    // rollover after every JWT-refresh hot-swap.
+                    val isCodecConfig =
+                        bufferInfo.flags and MediaCodec.BUFFER_FLAG_CODEC_CONFIG != 0
+                    if (isCodecConfig) {
+                        codec.releaseOutputBuffer(outputIndex, false)
+                        if (!loggedCsdSkip) {
+                            loggedCsdSkip = true
+                            com.vitorpamplona.quartz.utils.Log.d("NestTx") {
+                                "MediaCodecOpusEncoder skipped ${bufferInfo.size}-byte CODEC_CONFIG (OpusHead/OpusTags) — not an audio frame"
+                            }
+                        }
+                        // Don't return; loop to find the next output
+                        // buffer (the next dequeue may be the actual
+                        // first audio frame).
+                        continue
+                    }
                     val opus = ByteArray(bufferInfo.size)
                     outputBuffer.position(bufferInfo.offset)
                     outputBuffer.limit(bufferInfo.offset + bufferInfo.size)