fix(nests): platform-side audio robustness — focus, AEC, route obs, network handover

Four follow-up fixes from the post-audit review (#4 / #5 / #6 / #7). #6 AcousticEchoCanceler / NoiseSuppressor / AGC on the AudioRecord session. The VOICE_COMMUNICATION input source engages the platform echo canceller automatically on most modern Android devices, but a small set of older / OEM-customised devices only attach AEC under MODE_IN_COMMUNICATION — which an audio-room app deliberately avoids. Attaching the standalone audiofx effects to the AudioRecord's session id covers those devices without rerouting through the call audio path. All three are best-effort and a no-op on devices where the source already engages them. #4 Real audio focus handling. The previous OnAudioFocusChangeListener was a no-op based on the assumption that the OS would auto-duck us; it doesn't (CONTENT_TYPE_SPEECH streams aren't auto-ducked). Inbound phone calls were mixing on top of room audio. - New `NestAudioFocusBus` (commons) — process-wide enum signal, decoupled from android.media so commons stays platform-free. - NestForegroundService translates AUDIOFOCUS_GAIN / LOSS_TRANSIENT* / LOSS into the bus enum. - NestViewModel observes the bus and silences both the listener playback (effective listen-mute = user OR focus) and the broadcast mic (effective mic-mute = user OR focus). User-visible mute states stay the user's choice so a focus regain restores them automatically. - The pipeline keeps running while focus is lost (decoder, capture, network) so unmute is sample-accurate when the call ends. #5 AudioDeviceCallback observability. Registers a callback in NestForegroundService that logs Bluetooth / wired / USB headset attach + detach with device type + name. Doesn't drive playback decisions — Android's auto-routing handles route swaps — but makes "audio cut out when I plugged in headphones" reports correlatable with a concrete event for the first time. #7 Network-change → fast reconnect. Without this, a Wi-Fi → cellular handover left the QUIC connection sitting on a now-dead socket until its PTO fired (~30 s) before the wrapper noticed. Now: - New `NestNetworkChangeBus` (commons) — collapses bursts of onLost/onAvailable into a single recycle event. - NestsListener + NestsSpeaker grow `recycleSession()` (default no-op); the reconnecting wrappers override to close the inner session so their orchestrator opens a fresh one. - NestForegroundService registers a default-network callback; suppresses the first onAvailable (registration callback) and only publishes on actual default-network changes. - NestViewModel observes the bus and calls recycleSession on both wrappers. The SubscribeHandle re-issuance pump (listener) and the hot-swap publisher pump (speaker) cut existing subscriptions / broadcasts onto the new session as soon as it lands — same paths the JWT-refresh recycle uses. - Manifest gains ACCESS_NETWORK_STATE for registerDefaultNetworkCallback.
2026-05-05 12:42:47 +00:00
parent e4e55d1df6
commit 6237c02c6f
10 changed files with 635 additions and 15 deletions
@@ -22,6 +22,10 @@ package com.vitorpamplona.nestsclient.audio

 import android.media.AudioRecord
 import android.media.MediaRecorder
+import android.media.audiofx.AcousticEchoCanceler
+import android.media.audiofx.AutomaticGainControl
+import android.media.audiofx.NoiseSuppressor
+import com.vitorpamplona.quartz.utils.Log
 import kotlinx.coroutines.Dispatchers
 import kotlinx.coroutines.withContext
 import android.media.AudioFormat as AndroidAudioFormat
@@ -32,6 +36,19 @@ import android.media.AudioFormat as AndroidAudioFormat
 * voice-chat libraries use, so it gets the platform's echo-cancellation and
 * noise-suppression filters when available.
 *
+ * **Audio effects (AEC / NS / AGC).** On most modern Android devices the
+ * `VOICE_COMMUNICATION` source is enough to engage the platform's echo
+ * canceller automatically. On a small set of older / OEM-customised
+ * devices it isn't — the AEC engine only attaches when the device is in
+ * `MODE_IN_COMMUNICATION`, which an audio-room app deliberately avoids
+ * driving (it reroutes everything through the call audio path and shows a
+ * "phone call" notification icon). To cover those devices without
+ * touching `AudioManager.mode` we explicitly attach the standalone
+ * [AcousticEchoCanceler] / [NoiseSuppressor] / [AutomaticGainControl]
+ * effects to the AudioRecord's session id. On devices where the source
+ * already engages them, attaching a second effect is a no-op (the
+ * platform deduplicates by session id) — so this is purely additive.
+ *
 * **Permission:** the caller is responsible for holding `RECORD_AUDIO` before
 * calling [start]; this class will throw [AudioException.Kind.DeviceUnavailable]
 * if the OS denies the resource.
@@ -40,6 +57,9 @@ class AudioRecordCapture(
    private val source: Int = MediaRecorder.AudioSource.VOICE_COMMUNICATION,
 ) : AudioCapture {
    private var record: AudioRecord? = null
+    private var aec: AcousticEchoCanceler? = null
+    private var ns: NoiseSuppressor? = null
+    private var agc: AutomaticGainControl? = null
    private var stopped = false

    override fun start() {
@@ -97,6 +117,34 @@ class AudioRecordCapture(
            )
        }
        record = rec
+        // Attach standalone audio effects to the session. This covers
+        // devices where the VOICE_COMMUNICATION source alone doesn't
+        // engage AEC (typically because they only attach AEC under
+        // MODE_IN_COMMUNICATION). All three are best-effort — a device
+        // that doesn't support an effect leaves it null without
+        // affecting capture.
+        attachAudioEffects(rec.audioSessionId)
+    }
+
+    private fun attachAudioEffects(sessionId: Int) {
+        if (AcousticEchoCanceler.isAvailable()) {
+            aec =
+                runCatching { AcousticEchoCanceler.create(sessionId)?.apply { enabled = true } }
+                    .onFailure { Log.w("NestTx") { "AcousticEchoCanceler.create failed: ${it.message}" } }
+                    .getOrNull()
+        }
+        if (NoiseSuppressor.isAvailable()) {
+            ns =
+                runCatching { NoiseSuppressor.create(sessionId)?.apply { enabled = true } }
+                    .onFailure { Log.w("NestTx") { "NoiseSuppressor.create failed: ${it.message}" } }
+                    .getOrNull()
+        }
+        if (AutomaticGainControl.isAvailable()) {
+            agc =
+                runCatching { AutomaticGainControl.create(sessionId)?.apply { enabled = true } }
+                    .onFailure { Log.w("NestTx") { "AutomaticGainControl.create failed: ${it.message}" } }
+                    .getOrNull()
+        }
    }

    override suspend fun readFrame(): ShortArray? {
@@ -128,6 +176,16 @@ class AudioRecordCapture(
    override fun stop() {
        if (stopped) return
        stopped = true
+        // Release the audio effects BEFORE the AudioRecord — they hold
+        // a session-id reference and the platform expects effect
+        // teardown to precede the AudioRecord.release() that frees
+        // the session.
+        runCatching { aec?.release() }
+        runCatching { ns?.release() }
+        runCatching { agc?.release() }
+        aec = null
+        ns = null
+        agc = null
        val rec = record ?: return
        record = null
        runCatching { rec.stop() }
@@ -116,6 +116,27 @@ interface NestsListener {
            "announces() is moq-lite-only; IETF listener has no announce-prefix flow.",
        )

+    /**
+     * Force the underlying transport / MoQ session to be torn down and
+     * a fresh one opened in its place — without permanently closing
+     * this listener. Used by the platform layer on a network change
+     * (Wi-Fi ↔ cellular handover) to skip the ~30 s QUIC PTO that
+     * would otherwise have to fire before the wrapper notices the
+     * old socket is dead.
+     *
+     * Default no-op for non-reconnecting implementations (raw
+     * [DefaultNestsListener] / [MoqLiteNestsListener]) — there's no
+     * orchestrator to drive a reconnect, so a force recycle would
+     * just close the listener. The reconnecting wrapper
+     * (`ReconnectingHandle` from [connectReconnectingNestsListener])
+     * overrides to close the active inner listener so its
+     * orchestrator opens a fresh session.
+     */
+    suspend fun recycleSession() {
+        // no-op — only the reconnecting wrapper has anywhere to
+        // recycle to.
+    }
+
    /** Tear down the MoQ session + underlying transport. Idempotent. */
    suspend fun close()
 }
@@ -66,6 +66,27 @@ interface NestsSpeaker {
     */
    suspend fun startBroadcasting(onLevel: (Float) -> Unit = { /* no-op */ }): BroadcastHandle

+    /**
+     * Force the underlying transport / MoQ session to be torn down and
+     * a fresh one opened in its place — without permanently closing
+     * this speaker. Mirror of [NestsListener.recycleSession]; same
+     * use case (network handover) and same default no-op for non-
+     * reconnecting implementations.
+     *
+     * The reconnecting wrapper from
+     * [connectReconnectingNestsSpeaker] overrides to close the
+     * inner speaker so its orchestrator opens a fresh session.
+     * For moq-lite speakers, the long-lived
+     * [com.vitorpamplona.nestsclient.audio.NestMoqLiteBroadcaster]
+     * keeps capturing through the recycle and the
+     * [BroadcastHandle.swapPublisher]-based pump retargets onto
+     * the new session's publisher with no audible gap.
+     */
+    suspend fun recycleSession() {
+        // no-op — only the reconnecting wrapper has anywhere to
+        // recycle to.
+    }
+
    /** Tear down the MoQ session + transport. Idempotent. */
    suspend fun close()
 }
@@ -420,6 +420,29 @@ private class ReconnectingHandle(
        )
    }

+    /**
+     * Force-close the active inner listener so the orchestrator's
+     * `terminalAwait` returns Closed and the next loop iteration opens
+     * a fresh session. Used by the platform layer on a network change
+     * (Wi-Fi ↔ cellular) — without this, the wrapper would have to
+     * wait for the QUIC PTO to fire on the now-dead old socket
+     * (~30 s of silence) before noticing.
+     *
+     * The SubscribeHandle re-issuance pump cuts existing subs over to
+     * the new session as soon as it's Connected — same path the
+     * JWT-refresh recycle uses — so the consumer-facing
+     * [SubscribeHandle.objects] flow keeps emitting once the new
+     * session lands.
+     */
+    override suspend fun recycleSession() {
+        val current = activeListener.value ?: return
+        // Best-effort. If the close races a concurrent reconnect path
+        // (extremely rare — we only call this on deliberate user /
+        // platform signals), the orchestrator absorbs both via its
+        // existing terminal-state handling.
+        runCatching { current.close() }
+    }
+
    override suspend fun close() {
        orchestrator.cancel()
        runCatching { activeListener.value?.close() }
@@ -358,6 +358,17 @@ private class ReconnectingSpeakerHandle(
            handle
        }

+    /**
+     * Force-close the active inner speaker so the orchestrator opens
+     * a fresh session against the (presumably new) network. Used by
+     * the platform layer on a network change. Mirror of the listener
+     * wrapper's [com.vitorpamplona.nestsclient.NestsListener.recycleSession].
+     */
+    override suspend fun recycleSession() {
+        val current = activeSpeaker.value ?: return
+        runCatching { current.close() }
+    }
+
    override suspend fun close() {
        orchestrator.cancel()
        runCatching { activeBroadcast?.close() }