fix(nests): platform-side audio robustness — focus, AEC, route obs, network handover

Four follow-up fixes from the post-audit review (#4 / #5 / #6 / #7).

#6 AcousticEchoCanceler / NoiseSuppressor / AGC on the AudioRecord
   session. The VOICE_COMMUNICATION input source engages the platform
   echo canceller automatically on most modern Android devices, but a
   small set of older / OEM-customised devices only attach AEC under
   MODE_IN_COMMUNICATION — which an audio-room app deliberately
   avoids. Attaching the standalone audiofx effects to the
   AudioRecord's session id covers those devices without rerouting
   through the call audio path. All three are best-effort and a no-op
   on devices where the source already engages them.

#4 Real audio focus handling. The previous OnAudioFocusChangeListener
   was a no-op based on the assumption that the OS would auto-duck
   us; it doesn't (CONTENT_TYPE_SPEECH streams aren't auto-ducked).
   Inbound phone calls were mixing on top of room audio.
   - New `NestAudioFocusBus` (commons) — process-wide enum signal,
     decoupled from android.media so commons stays platform-free.
   - NestForegroundService translates AUDIOFOCUS_GAIN / LOSS_TRANSIENT*
     / LOSS into the bus enum.
   - NestViewModel observes the bus and silences both the listener
     playback (effective listen-mute = user OR focus) and the
     broadcast mic (effective mic-mute = user OR focus). User-visible
     mute states stay the user's choice so a focus regain restores
     them automatically.
   - The pipeline keeps running while focus is lost (decoder, capture,
     network) so unmute is sample-accurate when the call ends.

#5 AudioDeviceCallback observability. Registers a callback in
   NestForegroundService that logs Bluetooth / wired / USB headset
   attach + detach with device type + name. Doesn't drive playback
   decisions — Android's auto-routing handles route swaps — but
   makes "audio cut out when I plugged in headphones" reports
   correlatable with a concrete event for the first time.

#7 Network-change → fast reconnect. Without this, a Wi-Fi → cellular
   handover left the QUIC connection sitting on a now-dead socket
   until its PTO fired (~30 s) before the wrapper noticed. Now:
   - New `NestNetworkChangeBus` (commons) — collapses bursts of
     onLost/onAvailable into a single recycle event.
   - NestsListener + NestsSpeaker grow `recycleSession()` (default
     no-op); the reconnecting wrappers override to close the inner
     session so their orchestrator opens a fresh one.
   - NestForegroundService registers a default-network callback;
     suppresses the first onAvailable (registration callback)
     and only publishes on actual default-network changes.
   - NestViewModel observes the bus and calls recycleSession on
     both wrappers. The SubscribeHandle re-issuance pump (listener)
     and the hot-swap publisher pump (speaker) cut existing
     subscriptions / broadcasts onto the new session as soon as
     it lands — same paths the JWT-refresh recycle uses.
   - Manifest gains ACCESS_NETWORK_STATE for
     registerDefaultNetworkCallback.
This commit is contained in:
Claude
2026-05-05 12:42:47 +00:00
parent e4e55d1df6
commit 6237c02c6f
10 changed files with 635 additions and 15 deletions
@@ -0,0 +1,92 @@
/*
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.amethyst.commons.viewmodels
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
/**
* Process-wide audio-focus signal published by the platform-side audio
* focus listener (Android: `NestForegroundService`'s
* `OnAudioFocusChangeListener`) and consumed by every active
* [NestViewModel].
*
* Decoupled from the platform audio APIs so commons can stay free of
* `android.media.AudioManager` references — the platform layer
* translates focus-change codes into the small enum below before
* publishing.
*
* Why a singleton: there's at most one foreground audio-room session
* per process (the foreground service is unique), and every ViewModel
* wants to observe the same focus signal, so keying by some other
* dimension would just complicate the wiring. The bus survives
* activity / VM rotations naturally.
*/
object NestAudioFocusBus {
private val _state = MutableStateFlow(NestAudioFocusState.Granted)
/**
* The latest audio-focus state. Defaults to [NestAudioFocusState.Granted]
* — i.e. "we own playback" — so consumers don't have to special-case
* the boot-up window before the foreground service has a chance to
* register its listener.
*/
val state: StateFlow<NestAudioFocusState> = _state.asStateFlow()
/**
* Publish a new focus state. Only the platform-side listener calls
* this; consumers observe via [state].
*/
fun publish(newState: NestAudioFocusState) {
_state.value = newState
}
}
/**
* Audio-focus state translated from the platform's `AUDIOFOCUS_*` codes.
* Maps them onto the three actions an audio-room app actually cares
* about: keep playing, pause-because-something-else-is-playing, and
* stop-because-something-else-is-now-the-primary-audio-source.
*/
enum class NestAudioFocusState {
/** We own playback. Normal operation. */
Granted,
/**
* We've lost focus temporarily — typical triggers are an inbound
* phone call, a maps voice prompt, a system alarm. The
* [NestViewModel] reacts by silencing playback + the broadcast
* mic (the user-visible "muted" state stays unchanged so it
* restores on regain). Audio pipeline keeps running so resume
* is sample-accurate.
*/
TransientLoss,
/**
* Permanent focus loss — another app has taken over as the
* primary audio source for the foreseeable future. Same effective
* action as [TransientLoss] in v1 (silence both directions), but
* the distinction is preserved so future enhancements can tear
* down the audio device entirely on long-form loss.
*/
Loss,
}
@@ -0,0 +1,80 @@
/*
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.amethyst.commons.viewmodels
import kotlinx.coroutines.channels.BufferOverflow
import kotlinx.coroutines.flow.MutableSharedFlow
import kotlinx.coroutines.flow.SharedFlow
import kotlinx.coroutines.flow.asSharedFlow
/**
* Process-wide signal published by the platform-side connectivity
* listener (Android: `NestForegroundService`'s
* `ConnectivityManager.NetworkCallback`) and consumed by every active
* [NestViewModel] to recycle its [com.vitorpamplona.nestsclient.NestsListener]
* / [com.vitorpamplona.nestsclient.NestsSpeaker] sessions.
*
* Why this exists: when a phone hands over from Wi-Fi to cellular (or
* the other way), the local socket binding's source IP changes. The
* QUIC connection sitting on the previous socket isn't notified of
* the change — it'll keep retransmitting into the void until its PTO
* fires (`~30 s` for an idle connection). Without an external nudge
* the user hears 30 seconds of silence before the wrapper's reconnect
* loop notices the failure. With this bus, the wrapper recycles the
* QUIC session the moment the platform sees the network change,
* shrinking the audible gap to a single re-handshake (≈ 1 s on
* typical mobile networks).
*
* Decoupled from `android.net.ConnectivityManager` so commons stays
* platform-free — the platform layer translates `onAvailable` /
* `onLost` callbacks into a single `publish()` event before the
* VM observes it.
*/
object NestNetworkChangeBus {
/**
* `extraBufferCapacity = 1, DROP_OLDEST` — the consumer only cares
* that *a* network change happened, not how many. A burst of
* onLost / onAvailable calls during a Wi-Fi flap collapses to a
* single recycle event, which is exactly what we want (one
* handshake instead of N).
*/
private val _events =
MutableSharedFlow<Unit>(
replay = 0,
extraBufferCapacity = 1,
onBufferOverflow = BufferOverflow.DROP_OLDEST,
)
/**
* Hot flow of network-change events. Consumers ([NestViewModel])
* collect this and call `recycleSession()` on their listener +
* speaker.
*/
val events: SharedFlow<Unit> = _events.asSharedFlow()
/**
* Publish a network-change event. Only the platform-side listener
* calls this; consumers observe via [events].
*/
fun publish() {
_events.tryEmit(Unit)
}
}
@@ -282,6 +282,34 @@ class NestViewModel(
private var speakerStateJob: Job? = null
private var speakerConnectJob: Job? = null
/**
* Mirror of [NestAudioFocusBus.state] flipped to a boolean: `true`
* while another app holds audio focus (inbound phone call, Maps
* voice prompt, system alarm). Composed with the user-visible
* mute states (`NestUiState.isMuted` for listeners,
* `BroadcastUiState.Broadcasting.isMuted` for the speaker mic) via
* [effectiveListenMuted] / [effectiveMicMuted] so the user-chosen
* state is preserved across a focus loss + regain cycle.
*
* Maintained by [audioFocusObserverJob], started in [launchConnect]
* (so a never-connected VM doesn't subscribe needlessly).
*/
@Volatile private var focusMuted: Boolean = false
private var audioFocusObserverJob: Job? = null
/**
* Subscribed in [ensureNetworkChangeObserverStarted]; calls
* [com.vitorpamplona.nestsclient.NestsListener.recycleSession]
* (and the speaker counterpart) when the platform reports a
* default-network change. Without this, a Wi-Fi → cellular
* handover leaves the QUIC connection sitting on the now-dead
* socket until its PTO fires (~30 s) — the wrapper's reconnect
* loop only learns about the failure at PTO. With the bus
* nudge, recycle happens immediately and the audible gap
* shrinks to a single re-handshake.
*/
private var networkChangeObserverJob: Job? = null
/**
* Push the latest known speaker set from the room event. The user's
* own pubkey (when broadcasting) is filtered out so we don't subscribe
@@ -317,13 +345,97 @@ class NestViewModel(
teardown(targetState = ConnectionUiState.Idle, finalCleanup = false)
}
ensureAudioFocusObserverStarted()
ensureNetworkChangeObserverStarted()
launchConnect()
}
/**
* Subscribe to [NestAudioFocusBus] for the lifetime of this VM (or
* until [teardown] cancels). On loss, mute every active listener
* + the broadcast mic without touching the user-visible mute
* states; on regain, restore the user's choice. Idempotent —
* a second call while the job is alive is a no-op.
*/
private fun ensureAudioFocusObserverStarted() {
if (audioFocusObserverJob?.isActive == true) return
audioFocusObserverJob =
viewModelScope.launch {
NestAudioFocusBus.state.collect { focusState ->
if (closed) return@collect
val newFocusMuted = focusState != NestAudioFocusState.Granted
if (newFocusMuted == focusMuted) return@collect
focusMuted = newFocusMuted
// Re-apply both directions. effectiveListenMuted()
// and the per-broadcast effective rebuild below
// both consume the just-updated [focusMuted] field.
applyEffectiveListenMute()
applyEffectiveMicMute()
}
}
}
/**
* Subscribe to [NestNetworkChangeBus] for the lifetime of this VM
* (or until [teardown]). On a default-network change, ask both
* the listener and the speaker wrappers to recycle their
* underlying sessions — much faster than waiting for the QUIC
* PTO on a dead socket. Idempotent.
*/
private fun ensureNetworkChangeObserverStarted() {
if (networkChangeObserverJob?.isActive == true) return
networkChangeObserverJob =
viewModelScope.launch {
NestNetworkChangeBus.events.collect {
if (closed) return@collect
// Best-effort. The wrapper-level recycleSession is a
// no-op on raw / non-reconnecting implementations
// (test fakes, IETF reference) so a network change
// there is silently ignored — the production paths
// wrap with [connectReconnectingNestsListener] /
// [connectReconnectingNestsSpeaker] which both
// override the hook to close the inner session.
listener?.runCatching { recycleSession() }
speaker?.runCatching { recycleSession() }
}
}
}
/**
* Push the effective mic mute (`user choice OR focus-loss`) onto
* the live broadcast handle, if any. The user-visible
* [BroadcastUiState.Broadcasting.isMuted] stays the user's choice
* — UI only reflects that, never the focus-driven temporary
* silencing — so a focus regain is invisible to the user.
*/
private fun applyEffectiveMicMute() {
val handle = broadcastHandle ?: return
val ui = _uiState.value.broadcast
val userMuted = (ui as? BroadcastUiState.Broadcasting)?.isMuted ?: false
val effective = userMuted || focusMuted
viewModelScope.launch {
runCatching { handle.setMuted(effective) }
}
}
fun setMuted(muted: Boolean) {
if (closed) return
_uiState.update { it.copy(isMuted = muted) }
activeSubscriptions.values.forEach { it.player?.setMutedSafe(muted) }
applyEffectiveListenMute()
}
/**
* Effective listener mute = user-chosen mute (from the talk-bar)
* OR audio-focus loss (phone call / nav prompt). Applied to every
* active player. The user-visible state (`NestUiState.isMuted`)
* stays the user's choice, so a focus regain restores it
* automatically.
*/
private fun effectiveListenMuted(): Boolean = _uiState.value.isMuted || focusMuted
private fun applyEffectiveListenMute() {
val effective = effectiveListenMuted()
activeSubscriptions.values.forEach { it.player?.setMutedSafe(effective) }
}
/**
@@ -523,7 +635,13 @@ class NestViewModel(
if (closed) return
val handle = broadcastHandle ?: return
viewModelScope.launch {
val result = handle.runCatching { setMuted(muted) }
// Apply (user-mute OR audio-focus-loss) to the wire so the
// mic stays silent through a phone call without us losing
// the user's intent. UI then reflects the user choice (not
// the focus-effective state) so a focus-driven mute doesn't
// visually flip the talk button.
val effective = muted || focusMuted
val result = handle.runCatching { setMuted(effective) }
if (closed) return@launch
_uiState.update {
val current = it.broadcast
@@ -866,7 +984,12 @@ class NestViewModel(
throw t
}
try {
val isMuted = _uiState.value.isMuted
// Apply (user-mute OR audio-focus-loss) so a speaker
// that comes on stage during a phone call attaches
// already silenced. The focus observer re-runs
// applyEffectiveListenMute() on regain, restoring the
// user's intent.
val isMuted = effectiveListenMuted()
val isHushed = pubkey in _uiState.value.locallyHushed
val roomPlayer =
NestPlayer(
@@ -940,6 +1063,20 @@ class NestViewModel(
announcesJob = null
levelEmitterJob?.cancel()
levelEmitterJob = null
// Audio-focus observation only ends on the final teardown —
// a transient disconnect+reconnect (user retry, room swap)
// keeps the bus subscription alive so a focus loss that
// happens during the gap is still picked up. The observer
// is idempotent under [ensureAudioFocusObserverStarted], so
// the next [connect] call is safe whether or not we
// cancelled here.
if (finalCleanup) {
audioFocusObserverJob?.cancel()
audioFocusObserverJob = null
focusMuted = false
networkChangeObserverJob?.cancel()
networkChangeObserverJob = null
}
rawAudioLevels.clear()
if (_audioLevels.value.isNotEmpty()) {
_audioLevels.value = emptyMap()