fix(nests): platform-side audio robustness — focus, AEC, route obs, network handover
Four follow-up fixes from the post-audit review (#4 / #5 / #6 / #7). #6 AcousticEchoCanceler / NoiseSuppressor / AGC on the AudioRecord session. The VOICE_COMMUNICATION input source engages the platform echo canceller automatically on most modern Android devices, but a small set of older / OEM-customised devices only attach AEC under MODE_IN_COMMUNICATION — which an audio-room app deliberately avoids. Attaching the standalone audiofx effects to the AudioRecord's session id covers those devices without rerouting through the call audio path. All three are best-effort and a no-op on devices where the source already engages them. #4 Real audio focus handling. The previous OnAudioFocusChangeListener was a no-op based on the assumption that the OS would auto-duck us; it doesn't (CONTENT_TYPE_SPEECH streams aren't auto-ducked). Inbound phone calls were mixing on top of room audio. - New `NestAudioFocusBus` (commons) — process-wide enum signal, decoupled from android.media so commons stays platform-free. - NestForegroundService translates AUDIOFOCUS_GAIN / LOSS_TRANSIENT* / LOSS into the bus enum. - NestViewModel observes the bus and silences both the listener playback (effective listen-mute = user OR focus) and the broadcast mic (effective mic-mute = user OR focus). User-visible mute states stay the user's choice so a focus regain restores them automatically. - The pipeline keeps running while focus is lost (decoder, capture, network) so unmute is sample-accurate when the call ends. #5 AudioDeviceCallback observability. Registers a callback in NestForegroundService that logs Bluetooth / wired / USB headset attach + detach with device type + name. Doesn't drive playback decisions — Android's auto-routing handles route swaps — but makes "audio cut out when I plugged in headphones" reports correlatable with a concrete event for the first time. #7 Network-change → fast reconnect. Without this, a Wi-Fi → cellular handover left the QUIC connection sitting on a now-dead socket until its PTO fired (~30 s) before the wrapper noticed. Now: - New `NestNetworkChangeBus` (commons) — collapses bursts of onLost/onAvailable into a single recycle event. - NestsListener + NestsSpeaker grow `recycleSession()` (default no-op); the reconnecting wrappers override to close the inner session so their orchestrator opens a fresh one. - NestForegroundService registers a default-network callback; suppresses the first onAvailable (registration callback) and only publishes on actual default-network changes. - NestViewModel observes the bus and calls recycleSession on both wrappers. The SubscribeHandle re-issuance pump (listener) and the hot-swap publisher pump (speaker) cut existing subscriptions / broadcasts onto the new session as soon as it lands — same paths the JWT-refresh recycle uses. - Manifest gains ACCESS_NETWORK_STATE for registerDefaultNetworkCallback.
This commit is contained in:
+92
@@ -0,0 +1,92 @@
|
||||
/*
|
||||
* Copyright (c) 2025 Vitor Pamplona
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||
* this software and associated documentation files (the "Software"), to deal in
|
||||
* the Software without restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
|
||||
* Software, and to permit persons to whom the Software is furnished to do so,
|
||||
* subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in all
|
||||
* copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
|
||||
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
package com.vitorpamplona.amethyst.commons.viewmodels
|
||||
|
||||
import kotlinx.coroutines.flow.MutableStateFlow
|
||||
import kotlinx.coroutines.flow.StateFlow
|
||||
import kotlinx.coroutines.flow.asStateFlow
|
||||
|
||||
/**
|
||||
* Process-wide audio-focus signal published by the platform-side audio
|
||||
* focus listener (Android: `NestForegroundService`'s
|
||||
* `OnAudioFocusChangeListener`) and consumed by every active
|
||||
* [NestViewModel].
|
||||
*
|
||||
* Decoupled from the platform audio APIs so commons can stay free of
|
||||
* `android.media.AudioManager` references — the platform layer
|
||||
* translates focus-change codes into the small enum below before
|
||||
* publishing.
|
||||
*
|
||||
* Why a singleton: there's at most one foreground audio-room session
|
||||
* per process (the foreground service is unique), and every ViewModel
|
||||
* wants to observe the same focus signal, so keying by some other
|
||||
* dimension would just complicate the wiring. The bus survives
|
||||
* activity / VM rotations naturally.
|
||||
*/
|
||||
object NestAudioFocusBus {
|
||||
private val _state = MutableStateFlow(NestAudioFocusState.Granted)
|
||||
|
||||
/**
|
||||
* The latest audio-focus state. Defaults to [NestAudioFocusState.Granted]
|
||||
* — i.e. "we own playback" — so consumers don't have to special-case
|
||||
* the boot-up window before the foreground service has a chance to
|
||||
* register its listener.
|
||||
*/
|
||||
val state: StateFlow<NestAudioFocusState> = _state.asStateFlow()
|
||||
|
||||
/**
|
||||
* Publish a new focus state. Only the platform-side listener calls
|
||||
* this; consumers observe via [state].
|
||||
*/
|
||||
fun publish(newState: NestAudioFocusState) {
|
||||
_state.value = newState
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Audio-focus state translated from the platform's `AUDIOFOCUS_*` codes.
|
||||
* Maps them onto the three actions an audio-room app actually cares
|
||||
* about: keep playing, pause-because-something-else-is-playing, and
|
||||
* stop-because-something-else-is-now-the-primary-audio-source.
|
||||
*/
|
||||
enum class NestAudioFocusState {
|
||||
/** We own playback. Normal operation. */
|
||||
Granted,
|
||||
|
||||
/**
|
||||
* We've lost focus temporarily — typical triggers are an inbound
|
||||
* phone call, a maps voice prompt, a system alarm. The
|
||||
* [NestViewModel] reacts by silencing playback + the broadcast
|
||||
* mic (the user-visible "muted" state stays unchanged so it
|
||||
* restores on regain). Audio pipeline keeps running so resume
|
||||
* is sample-accurate.
|
||||
*/
|
||||
TransientLoss,
|
||||
|
||||
/**
|
||||
* Permanent focus loss — another app has taken over as the
|
||||
* primary audio source for the foreseeable future. Same effective
|
||||
* action as [TransientLoss] in v1 (silence both directions), but
|
||||
* the distinction is preserved so future enhancements can tear
|
||||
* down the audio device entirely on long-form loss.
|
||||
*/
|
||||
Loss,
|
||||
}
|
||||
+80
@@ -0,0 +1,80 @@
|
||||
/*
|
||||
* Copyright (c) 2025 Vitor Pamplona
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||
* this software and associated documentation files (the "Software"), to deal in
|
||||
* the Software without restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
|
||||
* Software, and to permit persons to whom the Software is furnished to do so,
|
||||
* subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in all
|
||||
* copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
|
||||
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
package com.vitorpamplona.amethyst.commons.viewmodels
|
||||
|
||||
import kotlinx.coroutines.channels.BufferOverflow
|
||||
import kotlinx.coroutines.flow.MutableSharedFlow
|
||||
import kotlinx.coroutines.flow.SharedFlow
|
||||
import kotlinx.coroutines.flow.asSharedFlow
|
||||
|
||||
/**
|
||||
* Process-wide signal published by the platform-side connectivity
|
||||
* listener (Android: `NestForegroundService`'s
|
||||
* `ConnectivityManager.NetworkCallback`) and consumed by every active
|
||||
* [NestViewModel] to recycle its [com.vitorpamplona.nestsclient.NestsListener]
|
||||
* / [com.vitorpamplona.nestsclient.NestsSpeaker] sessions.
|
||||
*
|
||||
* Why this exists: when a phone hands over from Wi-Fi to cellular (or
|
||||
* the other way), the local socket binding's source IP changes. The
|
||||
* QUIC connection sitting on the previous socket isn't notified of
|
||||
* the change — it'll keep retransmitting into the void until its PTO
|
||||
* fires (`~30 s` for an idle connection). Without an external nudge
|
||||
* the user hears 30 seconds of silence before the wrapper's reconnect
|
||||
* loop notices the failure. With this bus, the wrapper recycles the
|
||||
* QUIC session the moment the platform sees the network change,
|
||||
* shrinking the audible gap to a single re-handshake (≈ 1 s on
|
||||
* typical mobile networks).
|
||||
*
|
||||
* Decoupled from `android.net.ConnectivityManager` so commons stays
|
||||
* platform-free — the platform layer translates `onAvailable` /
|
||||
* `onLost` callbacks into a single `publish()` event before the
|
||||
* VM observes it.
|
||||
*/
|
||||
object NestNetworkChangeBus {
|
||||
/**
|
||||
* `extraBufferCapacity = 1, DROP_OLDEST` — the consumer only cares
|
||||
* that *a* network change happened, not how many. A burst of
|
||||
* onLost / onAvailable calls during a Wi-Fi flap collapses to a
|
||||
* single recycle event, which is exactly what we want (one
|
||||
* handshake instead of N).
|
||||
*/
|
||||
private val _events =
|
||||
MutableSharedFlow<Unit>(
|
||||
replay = 0,
|
||||
extraBufferCapacity = 1,
|
||||
onBufferOverflow = BufferOverflow.DROP_OLDEST,
|
||||
)
|
||||
|
||||
/**
|
||||
* Hot flow of network-change events. Consumers ([NestViewModel])
|
||||
* collect this and call `recycleSession()` on their listener +
|
||||
* speaker.
|
||||
*/
|
||||
val events: SharedFlow<Unit> = _events.asSharedFlow()
|
||||
|
||||
/**
|
||||
* Publish a network-change event. Only the platform-side listener
|
||||
* calls this; consumers observe via [events].
|
||||
*/
|
||||
fun publish() {
|
||||
_events.tryEmit(Unit)
|
||||
}
|
||||
}
|
||||
+140
-3
@@ -282,6 +282,34 @@ class NestViewModel(
|
||||
private var speakerStateJob: Job? = null
|
||||
private var speakerConnectJob: Job? = null
|
||||
|
||||
/**
|
||||
* Mirror of [NestAudioFocusBus.state] flipped to a boolean: `true`
|
||||
* while another app holds audio focus (inbound phone call, Maps
|
||||
* voice prompt, system alarm). Composed with the user-visible
|
||||
* mute states (`NestUiState.isMuted` for listeners,
|
||||
* `BroadcastUiState.Broadcasting.isMuted` for the speaker mic) via
|
||||
* [effectiveListenMuted] / [effectiveMicMuted] so the user-chosen
|
||||
* state is preserved across a focus loss + regain cycle.
|
||||
*
|
||||
* Maintained by [audioFocusObserverJob], started in [launchConnect]
|
||||
* (so a never-connected VM doesn't subscribe needlessly).
|
||||
*/
|
||||
@Volatile private var focusMuted: Boolean = false
|
||||
private var audioFocusObserverJob: Job? = null
|
||||
|
||||
/**
|
||||
* Subscribed in [ensureNetworkChangeObserverStarted]; calls
|
||||
* [com.vitorpamplona.nestsclient.NestsListener.recycleSession]
|
||||
* (and the speaker counterpart) when the platform reports a
|
||||
* default-network change. Without this, a Wi-Fi → cellular
|
||||
* handover leaves the QUIC connection sitting on the now-dead
|
||||
* socket until its PTO fires (~30 s) — the wrapper's reconnect
|
||||
* loop only learns about the failure at PTO. With the bus
|
||||
* nudge, recycle happens immediately and the audible gap
|
||||
* shrinks to a single re-handshake.
|
||||
*/
|
||||
private var networkChangeObserverJob: Job? = null
|
||||
|
||||
/**
|
||||
* Push the latest known speaker set from the room event. The user's
|
||||
* own pubkey (when broadcasting) is filtered out so we don't subscribe
|
||||
@@ -317,13 +345,97 @@ class NestViewModel(
|
||||
teardown(targetState = ConnectionUiState.Idle, finalCleanup = false)
|
||||
}
|
||||
|
||||
ensureAudioFocusObserverStarted()
|
||||
ensureNetworkChangeObserverStarted()
|
||||
launchConnect()
|
||||
}
|
||||
|
||||
/**
|
||||
* Subscribe to [NestAudioFocusBus] for the lifetime of this VM (or
|
||||
* until [teardown] cancels). On loss, mute every active listener
|
||||
* + the broadcast mic without touching the user-visible mute
|
||||
* states; on regain, restore the user's choice. Idempotent —
|
||||
* a second call while the job is alive is a no-op.
|
||||
*/
|
||||
private fun ensureAudioFocusObserverStarted() {
|
||||
if (audioFocusObserverJob?.isActive == true) return
|
||||
audioFocusObserverJob =
|
||||
viewModelScope.launch {
|
||||
NestAudioFocusBus.state.collect { focusState ->
|
||||
if (closed) return@collect
|
||||
val newFocusMuted = focusState != NestAudioFocusState.Granted
|
||||
if (newFocusMuted == focusMuted) return@collect
|
||||
focusMuted = newFocusMuted
|
||||
// Re-apply both directions. effectiveListenMuted()
|
||||
// and the per-broadcast effective rebuild below
|
||||
// both consume the just-updated [focusMuted] field.
|
||||
applyEffectiveListenMute()
|
||||
applyEffectiveMicMute()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Subscribe to [NestNetworkChangeBus] for the lifetime of this VM
|
||||
* (or until [teardown]). On a default-network change, ask both
|
||||
* the listener and the speaker wrappers to recycle their
|
||||
* underlying sessions — much faster than waiting for the QUIC
|
||||
* PTO on a dead socket. Idempotent.
|
||||
*/
|
||||
private fun ensureNetworkChangeObserverStarted() {
|
||||
if (networkChangeObserverJob?.isActive == true) return
|
||||
networkChangeObserverJob =
|
||||
viewModelScope.launch {
|
||||
NestNetworkChangeBus.events.collect {
|
||||
if (closed) return@collect
|
||||
// Best-effort. The wrapper-level recycleSession is a
|
||||
// no-op on raw / non-reconnecting implementations
|
||||
// (test fakes, IETF reference) so a network change
|
||||
// there is silently ignored — the production paths
|
||||
// wrap with [connectReconnectingNestsListener] /
|
||||
// [connectReconnectingNestsSpeaker] which both
|
||||
// override the hook to close the inner session.
|
||||
listener?.runCatching { recycleSession() }
|
||||
speaker?.runCatching { recycleSession() }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Push the effective mic mute (`user choice OR focus-loss`) onto
|
||||
* the live broadcast handle, if any. The user-visible
|
||||
* [BroadcastUiState.Broadcasting.isMuted] stays the user's choice
|
||||
* — UI only reflects that, never the focus-driven temporary
|
||||
* silencing — so a focus regain is invisible to the user.
|
||||
*/
|
||||
private fun applyEffectiveMicMute() {
|
||||
val handle = broadcastHandle ?: return
|
||||
val ui = _uiState.value.broadcast
|
||||
val userMuted = (ui as? BroadcastUiState.Broadcasting)?.isMuted ?: false
|
||||
val effective = userMuted || focusMuted
|
||||
viewModelScope.launch {
|
||||
runCatching { handle.setMuted(effective) }
|
||||
}
|
||||
}
|
||||
|
||||
fun setMuted(muted: Boolean) {
|
||||
if (closed) return
|
||||
_uiState.update { it.copy(isMuted = muted) }
|
||||
activeSubscriptions.values.forEach { it.player?.setMutedSafe(muted) }
|
||||
applyEffectiveListenMute()
|
||||
}
|
||||
|
||||
/**
|
||||
* Effective listener mute = user-chosen mute (from the talk-bar)
|
||||
* OR audio-focus loss (phone call / nav prompt). Applied to every
|
||||
* active player. The user-visible state (`NestUiState.isMuted`)
|
||||
* stays the user's choice, so a focus regain restores it
|
||||
* automatically.
|
||||
*/
|
||||
private fun effectiveListenMuted(): Boolean = _uiState.value.isMuted || focusMuted
|
||||
|
||||
private fun applyEffectiveListenMute() {
|
||||
val effective = effectiveListenMuted()
|
||||
activeSubscriptions.values.forEach { it.player?.setMutedSafe(effective) }
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -523,7 +635,13 @@ class NestViewModel(
|
||||
if (closed) return
|
||||
val handle = broadcastHandle ?: return
|
||||
viewModelScope.launch {
|
||||
val result = handle.runCatching { setMuted(muted) }
|
||||
// Apply (user-mute OR audio-focus-loss) to the wire so the
|
||||
// mic stays silent through a phone call without us losing
|
||||
// the user's intent. UI then reflects the user choice (not
|
||||
// the focus-effective state) so a focus-driven mute doesn't
|
||||
// visually flip the talk button.
|
||||
val effective = muted || focusMuted
|
||||
val result = handle.runCatching { setMuted(effective) }
|
||||
if (closed) return@launch
|
||||
_uiState.update {
|
||||
val current = it.broadcast
|
||||
@@ -866,7 +984,12 @@ class NestViewModel(
|
||||
throw t
|
||||
}
|
||||
try {
|
||||
val isMuted = _uiState.value.isMuted
|
||||
// Apply (user-mute OR audio-focus-loss) so a speaker
|
||||
// that comes on stage during a phone call attaches
|
||||
// already silenced. The focus observer re-runs
|
||||
// applyEffectiveListenMute() on regain, restoring the
|
||||
// user's intent.
|
||||
val isMuted = effectiveListenMuted()
|
||||
val isHushed = pubkey in _uiState.value.locallyHushed
|
||||
val roomPlayer =
|
||||
NestPlayer(
|
||||
@@ -940,6 +1063,20 @@ class NestViewModel(
|
||||
announcesJob = null
|
||||
levelEmitterJob?.cancel()
|
||||
levelEmitterJob = null
|
||||
// Audio-focus observation only ends on the final teardown —
|
||||
// a transient disconnect+reconnect (user retry, room swap)
|
||||
// keeps the bus subscription alive so a focus loss that
|
||||
// happens during the gap is still picked up. The observer
|
||||
// is idempotent under [ensureAudioFocusObserverStarted], so
|
||||
// the next [connect] call is safe whether or not we
|
||||
// cancelled here.
|
||||
if (finalCleanup) {
|
||||
audioFocusObserverJob?.cancel()
|
||||
audioFocusObserverJob = null
|
||||
focusMuted = false
|
||||
networkChangeObserverJob?.cancel()
|
||||
networkChangeObserverJob = null
|
||||
}
|
||||
rawAudioLevels.clear()
|
||||
if (_audioLevels.value.isNotEmpty()) {
|
||||
_audioLevels.value = emptyMap()
|
||||
|
||||
Reference in New Issue
Block a user