fix(nests): platform-side audio robustness — focus, AEC, route obs, network handover

Four follow-up fixes from the post-audit review (#4 / #5 / #6 / #7).

#6 AcousticEchoCanceler / NoiseSuppressor / AGC on the AudioRecord
   session. The VOICE_COMMUNICATION input source engages the platform
   echo canceller automatically on most modern Android devices, but a
   small set of older / OEM-customised devices only attach AEC under
   MODE_IN_COMMUNICATION — which an audio-room app deliberately
   avoids. Attaching the standalone audiofx effects to the
   AudioRecord's session id covers those devices without rerouting
   through the call audio path. All three are best-effort and a no-op
   on devices where the source already engages them.

#4 Real audio focus handling. The previous OnAudioFocusChangeListener
   was a no-op based on the assumption that the OS would auto-duck
   us; it doesn't (CONTENT_TYPE_SPEECH streams aren't auto-ducked).
   Inbound phone calls were mixing on top of room audio.
   - New `NestAudioFocusBus` (commons) — process-wide enum signal,
     decoupled from android.media so commons stays platform-free.
   - NestForegroundService translates AUDIOFOCUS_GAIN / LOSS_TRANSIENT*
     / LOSS into the bus enum.
   - NestViewModel observes the bus and silences both the listener
     playback (effective listen-mute = user OR focus) and the
     broadcast mic (effective mic-mute = user OR focus). User-visible
     mute states stay the user's choice so a focus regain restores
     them automatically.
   - The pipeline keeps running while focus is lost (decoder, capture,
     network) so unmute is sample-accurate when the call ends.

#5 AudioDeviceCallback observability. Registers a callback in
   NestForegroundService that logs Bluetooth / wired / USB headset
   attach + detach with device type + name. Doesn't drive playback
   decisions — Android's auto-routing handles route swaps — but
   makes "audio cut out when I plugged in headphones" reports
   correlatable with a concrete event for the first time.

#7 Network-change → fast reconnect. Without this, a Wi-Fi → cellular
   handover left the QUIC connection sitting on a now-dead socket
   until its PTO fired (~30 s) before the wrapper noticed. Now:
   - New `NestNetworkChangeBus` (commons) — collapses bursts of
     onLost/onAvailable into a single recycle event.
   - NestsListener + NestsSpeaker grow `recycleSession()` (default
     no-op); the reconnecting wrappers override to close the inner
     session so their orchestrator opens a fresh one.
   - NestForegroundService registers a default-network callback;
     suppresses the first onAvailable (registration callback)
     and only publishes on actual default-network changes.
   - NestViewModel observes the bus and calls recycleSession on
     both wrappers. The SubscribeHandle re-issuance pump (listener)
     and the hot-swap publisher pump (speaker) cut existing
     subscriptions / broadcasts onto the new session as soon as
     it lands — same paths the JWT-refresh recycle uses.
   - Manifest gains ACCESS_NETWORK_STATE for
     registerDefaultNetworkCallback.
This commit is contained in:
Claude
2026-05-05 12:42:47 +00:00
parent e4e55d1df6
commit 6237c02c6f
10 changed files with 635 additions and 15 deletions
+5
View File
@@ -37,6 +37,11 @@
<!-- To know receive notifications when the app connects/disconnects from the web -->
<uses-permission android:name="android.permission.CHANGE_NETWORK_STATE" />
<!-- For ConnectivityManager.registerDefaultNetworkCallback used by NestForegroundService
to detect Wi-Fi ↔ cellular handover and recycle the active QUIC session promptly
(otherwise the wrapper waits ~30s for QUIC PTO before reconnecting). -->
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<!-- Audio/Video Playback -->
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK" />
@@ -28,13 +28,23 @@ import android.app.Service
import android.content.Context
import android.content.Intent
import android.content.pm.ServiceInfo
import android.media.AudioDeviceCallback
import android.media.AudioDeviceInfo
import android.media.AudioFocusRequest
import android.media.AudioManager
import android.net.ConnectivityManager
import android.net.Network
import android.os.Build
import android.os.IBinder
import android.os.PowerManager
import androidx.core.app.NotificationCompat
import androidx.core.content.ContextCompat
import com.vitorpamplona.amethyst.R
import com.vitorpamplona.amethyst.commons.viewmodels.NestAudioFocusBus
import com.vitorpamplona.amethyst.commons.viewmodels.NestAudioFocusState
import com.vitorpamplona.amethyst.commons.viewmodels.NestNetworkChangeBus
import com.vitorpamplona.amethyst.ui.MainActivity
import com.vitorpamplona.quartz.utils.Log
/**
* Process-anchor for an active audio-room session. Holds a partial wake-lock
@@ -57,7 +67,42 @@ import com.vitorpamplona.amethyst.ui.MainActivity
class NestForegroundService : Service() {
private var wakeLock: PowerManager.WakeLock? = null
private var promoted = false
private var audioFocusRequest: android.media.AudioFocusRequest? = null
private var audioFocusRequest: AudioFocusRequest? = null
/**
* Logs route changes (Bluetooth headset attach/detach, wired
* headset, USB audio, speakerphone) so field reports of
* "audio cut out when I plugged in headphones" can be correlated
* with a concrete device-add / device-remove event.
*
* Doesn't drive playback decisions — Android's AudioTrack +
* AudioRecord auto-route to whichever device the OS treats as
* active, so the brief silence on a route swap is unavoidable
* without going through the (intrusive) `MODE_IN_COMMUNICATION` +
* `setCommunicationDevice` flow that this service deliberately
* avoids. v1 ships observability only; future work could pause
* playback briefly across a route change to mask the swap.
*/
private val deviceCallback =
object : AudioDeviceCallback() {
override fun onAudioDevicesAdded(addedDevices: Array<out AudioDeviceInfo>?) {
addedDevices?.forEach { dev ->
Log.i("NestAudio") {
"audio device added: type=${dev.type} name='${dev.productName}' " +
"isSink=${dev.isSink} isSource=${dev.isSource}"
}
}
}
override fun onAudioDevicesRemoved(removedDevices: Array<out AudioDeviceInfo>?) {
removedDevices?.forEach { dev ->
Log.i("NestAudio") {
"audio device removed: type=${dev.type} name='${dev.productName}' " +
"isSink=${dev.isSink} isSource=${dev.isSource}"
}
}
}
}
override fun onCreate() {
super.onCreate()
@@ -67,13 +112,103 @@ class NestForegroundService : Service() {
.newWakeLock(PowerManager.PARTIAL_WAKE_LOCK, "amethyst:audio-room")
.apply { setReferenceCounted(false) }
requestAudioFocus()
registerAudioDeviceCallback()
registerNetworkCallback()
}
private fun registerAudioDeviceCallback() {
runCatching {
val mgr = getSystemService(Context.AUDIO_SERVICE) as AudioManager
// null Handler → callback runs on the main looper, which is
// the cheapest path for "log a line" handlers like this.
mgr.registerAudioDeviceCallback(deviceCallback, null)
}
}
private fun unregisterAudioDeviceCallback() {
runCatching {
val mgr = getSystemService(Context.AUDIO_SERVICE) as AudioManager
mgr.unregisterAudioDeviceCallback(deviceCallback)
}
}
/**
* Request transient-may-duck audio focus so an inbound phone
* call lowers the room volume cleanly instead of mixing two
* voices on top of each other. Acquired once per service
* lifetime; released in [onDestroy].
* Track the current default network so the NetworkCallback can
* distinguish "first onAvailable after register" (no-op) from "the
* default network just changed under us" (publish). Volatile because
* the callback fires on a binder thread; the publish path is
* lock-free via [NestNetworkChangeBus].
*/
@Volatile private var currentDefaultNetwork: Network? = null
/**
* Listens for the device's default-network changing (Wi-Fi ↔
* cellular handover, plane mode toggle, hotspot swap) and signals
* every active [com.vitorpamplona.amethyst.commons.viewmodels.NestViewModel]
* to recycle its QUIC session. Without this nudge the QUIC
* connection sitting on the now-dead socket would have to wait
* for its PTO (~30 s) before the wrapper notices — a long
* audible silence on every handover.
*
* The callback also fires once right after registration with the
* current default network — we suppress that emission so the VM
* doesn't recycle on every service start.
*/
private val networkCallback =
object : ConnectivityManager.NetworkCallback() {
override fun onAvailable(network: Network) {
val previous = currentDefaultNetwork
currentDefaultNetwork = network
if (previous != null && previous != network) {
Log.i("NestNet") {
"default network changed ($previous$network), recycling QUIC sessions"
}
NestNetworkChangeBus.publish()
}
}
override fun onLost(network: Network) {
if (currentDefaultNetwork == network) {
// We've lost the current default. Don't publish here —
// the next onAvailable (with the replacement network)
// will, and recycling NOW means the wrapper would try
// to handshake on no network at all and fail-then-
// backoff. Just clear so the next onAvailable is
// recognised as a change.
currentDefaultNetwork = null
}
}
}
private fun registerNetworkCallback() {
runCatching {
val mgr = getSystemService(Context.CONNECTIVITY_SERVICE) as ConnectivityManager
mgr.registerDefaultNetworkCallback(networkCallback)
}
}
private fun unregisterNetworkCallback() {
runCatching {
val mgr = getSystemService(Context.CONNECTIVITY_SERVICE) as ConnectivityManager
mgr.unregisterNetworkCallback(networkCallback)
}
currentDefaultNetwork = null
}
/**
* Request audio focus for the duration of the audio-room session
* and route the system's focus-change events into [NestAudioFocusBus]
* so every active [com.vitorpamplona.amethyst.commons.viewmodels.NestViewModel]
* can react.
*
* Why we actually handle the focus change (vs the previous no-op
* listener): the platform only auto-ducks streams that opted into
* auto-ducking (CONTENT_TYPE_MUSIC, etc.) — a `CONTENT_TYPE_SPEECH`
* stream is left alone, so without a real listener an inbound phone
* call would mix on top of the room audio and a Maps voice prompt
* would be inaudible against an active speaker. The bus carries
* the translated state to the VM, which silences the listener
* playback and the broadcast mic for the duration of the loss.
*
* Matches the playback `AudioAttributes` we set on `AudioTrack`
* in `AudioTrackPlayer` (USAGE_MEDIA + CONTENT_TYPE_SPEECH) so
@@ -82,7 +217,7 @@ class NestForegroundService : Service() {
*/
private fun requestAudioFocus() {
if (audioFocusRequest != null) return
val mgr = getSystemService(Context.AUDIO_SERVICE) as android.media.AudioManager
val mgr = getSystemService(Context.AUDIO_SERVICE) as AudioManager
val attrs =
android.media.AudioAttributes
.Builder()
@@ -90,24 +225,59 @@ class NestForegroundService : Service() {
.setContentType(android.media.AudioAttributes.CONTENT_TYPE_SPEECH)
.build()
val request =
android.media.AudioFocusRequest
.Builder(android.media.AudioManager.AUDIOFOCUS_GAIN)
AudioFocusRequest
.Builder(AudioManager.AUDIOFOCUS_GAIN)
.setAudioAttributes(attrs)
.setAcceptsDelayedFocusGain(false)
.setOnAudioFocusChangeListener({ /* no-op — duck-on-loss handled by the OS */ })
.build()
.setOnAudioFocusChangeListener { focusChange ->
NestAudioFocusBus.publish(translateFocusChange(focusChange))
}.build()
// Best-effort: a refused request just means the OS will
// duck/pause us based on its own policy. Don't fail the
// service start over a focus denial.
runCatching { mgr.requestAudioFocus(request) }
val granted = runCatching { mgr.requestAudioFocus(request) }.getOrNull()
// If the OS refused outright (rare — typically only when an
// active call is already in progress at the moment we start),
// publish TransientLoss immediately so the VM mutes from t=0
// rather than playing for ~50 ms before the listener fires.
// [AudioManager.AUDIOFOCUS_REQUEST_FAILED] = 0; granted = 1.
if (granted == AudioManager.AUDIOFOCUS_REQUEST_FAILED) {
NestAudioFocusBus.publish(NestAudioFocusState.TransientLoss)
} else {
NestAudioFocusBus.publish(NestAudioFocusState.Granted)
}
audioFocusRequest = request
}
private fun abandonAudioFocus() {
val request = audioFocusRequest ?: return
val mgr = getSystemService(Context.AUDIO_SERVICE) as android.media.AudioManager
val mgr = getSystemService(Context.AUDIO_SERVICE) as AudioManager
runCatching { mgr.abandonAudioFocusRequest(request) }
audioFocusRequest = null
// Reset to Granted so a future foreground-service start that
// happens before the next focus request lands doesn't inherit
// a stale "muted because we lost focus" state.
NestAudioFocusBus.publish(NestAudioFocusState.Granted)
}
private fun translateFocusChange(focusChange: Int): NestAudioFocusState =
when (focusChange) {
AudioManager.AUDIOFOCUS_GAIN,
AudioManager.AUDIOFOCUS_GAIN_TRANSIENT,
AudioManager.AUDIOFOCUS_GAIN_TRANSIENT_MAY_DUCK,
AudioManager.AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE,
-> NestAudioFocusState.Granted
AudioManager.AUDIOFOCUS_LOSS_TRANSIENT,
AudioManager.AUDIOFOCUS_LOSS_TRANSIENT_CAN_DUCK,
-> NestAudioFocusState.TransientLoss
AudioManager.AUDIOFOCUS_LOSS -> NestAudioFocusState.Loss
// Unknown future codes: be defensive and treat as Granted
// so a vendor-specific extension can't silently mute the
// room forever.
else -> NestAudioFocusState.Granted
}
override fun onStartCommand(
@@ -217,6 +387,8 @@ class NestForegroundService : Service() {
wakeLock?.takeIf { it.isHeld }?.release()
wakeLock = null
abandonAudioFocus()
unregisterAudioDeviceCallback()
unregisterNetworkCallback()
super.onDestroy()
}
@@ -0,0 +1,92 @@
/*
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.amethyst.commons.viewmodels
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
/**
* Process-wide audio-focus signal published by the platform-side audio
* focus listener (Android: `NestForegroundService`'s
* `OnAudioFocusChangeListener`) and consumed by every active
* [NestViewModel].
*
* Decoupled from the platform audio APIs so commons can stay free of
* `android.media.AudioManager` references — the platform layer
* translates focus-change codes into the small enum below before
* publishing.
*
* Why a singleton: there's at most one foreground audio-room session
* per process (the foreground service is unique), and every ViewModel
* wants to observe the same focus signal, so keying by some other
* dimension would just complicate the wiring. The bus survives
* activity / VM rotations naturally.
*/
object NestAudioFocusBus {
private val _state = MutableStateFlow(NestAudioFocusState.Granted)
/**
* The latest audio-focus state. Defaults to [NestAudioFocusState.Granted]
* — i.e. "we own playback" — so consumers don't have to special-case
* the boot-up window before the foreground service has a chance to
* register its listener.
*/
val state: StateFlow<NestAudioFocusState> = _state.asStateFlow()
/**
* Publish a new focus state. Only the platform-side listener calls
* this; consumers observe via [state].
*/
fun publish(newState: NestAudioFocusState) {
_state.value = newState
}
}
/**
* Audio-focus state translated from the platform's `AUDIOFOCUS_*` codes.
* Maps them onto the three actions an audio-room app actually cares
* about: keep playing, pause-because-something-else-is-playing, and
* stop-because-something-else-is-now-the-primary-audio-source.
*/
enum class NestAudioFocusState {
/** We own playback. Normal operation. */
Granted,
/**
* We've lost focus temporarily — typical triggers are an inbound
* phone call, a maps voice prompt, a system alarm. The
* [NestViewModel] reacts by silencing playback + the broadcast
* mic (the user-visible "muted" state stays unchanged so it
* restores on regain). Audio pipeline keeps running so resume
* is sample-accurate.
*/
TransientLoss,
/**
* Permanent focus loss — another app has taken over as the
* primary audio source for the foreseeable future. Same effective
* action as [TransientLoss] in v1 (silence both directions), but
* the distinction is preserved so future enhancements can tear
* down the audio device entirely on long-form loss.
*/
Loss,
}
@@ -0,0 +1,80 @@
/*
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.amethyst.commons.viewmodels
import kotlinx.coroutines.channels.BufferOverflow
import kotlinx.coroutines.flow.MutableSharedFlow
import kotlinx.coroutines.flow.SharedFlow
import kotlinx.coroutines.flow.asSharedFlow
/**
* Process-wide signal published by the platform-side connectivity
* listener (Android: `NestForegroundService`'s
* `ConnectivityManager.NetworkCallback`) and consumed by every active
* [NestViewModel] to recycle its [com.vitorpamplona.nestsclient.NestsListener]
* / [com.vitorpamplona.nestsclient.NestsSpeaker] sessions.
*
* Why this exists: when a phone hands over from Wi-Fi to cellular (or
* the other way), the local socket binding's source IP changes. The
* QUIC connection sitting on the previous socket isn't notified of
* the change — it'll keep retransmitting into the void until its PTO
* fires (`~30 s` for an idle connection). Without an external nudge
* the user hears 30 seconds of silence before the wrapper's reconnect
* loop notices the failure. With this bus, the wrapper recycles the
* QUIC session the moment the platform sees the network change,
* shrinking the audible gap to a single re-handshake (≈ 1 s on
* typical mobile networks).
*
* Decoupled from `android.net.ConnectivityManager` so commons stays
* platform-free — the platform layer translates `onAvailable` /
* `onLost` callbacks into a single `publish()` event before the
* VM observes it.
*/
object NestNetworkChangeBus {
/**
* `extraBufferCapacity = 1, DROP_OLDEST` — the consumer only cares
* that *a* network change happened, not how many. A burst of
* onLost / onAvailable calls during a Wi-Fi flap collapses to a
* single recycle event, which is exactly what we want (one
* handshake instead of N).
*/
private val _events =
MutableSharedFlow<Unit>(
replay = 0,
extraBufferCapacity = 1,
onBufferOverflow = BufferOverflow.DROP_OLDEST,
)
/**
* Hot flow of network-change events. Consumers ([NestViewModel])
* collect this and call `recycleSession()` on their listener +
* speaker.
*/
val events: SharedFlow<Unit> = _events.asSharedFlow()
/**
* Publish a network-change event. Only the platform-side listener
* calls this; consumers observe via [events].
*/
fun publish() {
_events.tryEmit(Unit)
}
}
@@ -282,6 +282,34 @@ class NestViewModel(
private var speakerStateJob: Job? = null
private var speakerConnectJob: Job? = null
/**
* Mirror of [NestAudioFocusBus.state] flipped to a boolean: `true`
* while another app holds audio focus (inbound phone call, Maps
* voice prompt, system alarm). Composed with the user-visible
* mute states (`NestUiState.isMuted` for listeners,
* `BroadcastUiState.Broadcasting.isMuted` for the speaker mic) via
* [effectiveListenMuted] / [effectiveMicMuted] so the user-chosen
* state is preserved across a focus loss + regain cycle.
*
* Maintained by [audioFocusObserverJob], started in [launchConnect]
* (so a never-connected VM doesn't subscribe needlessly).
*/
@Volatile private var focusMuted: Boolean = false
private var audioFocusObserverJob: Job? = null
/**
* Subscribed in [ensureNetworkChangeObserverStarted]; calls
* [com.vitorpamplona.nestsclient.NestsListener.recycleSession]
* (and the speaker counterpart) when the platform reports a
* default-network change. Without this, a Wi-Fi → cellular
* handover leaves the QUIC connection sitting on the now-dead
* socket until its PTO fires (~30 s) — the wrapper's reconnect
* loop only learns about the failure at PTO. With the bus
* nudge, recycle happens immediately and the audible gap
* shrinks to a single re-handshake.
*/
private var networkChangeObserverJob: Job? = null
/**
* Push the latest known speaker set from the room event. The user's
* own pubkey (when broadcasting) is filtered out so we don't subscribe
@@ -317,13 +345,97 @@ class NestViewModel(
teardown(targetState = ConnectionUiState.Idle, finalCleanup = false)
}
ensureAudioFocusObserverStarted()
ensureNetworkChangeObserverStarted()
launchConnect()
}
/**
* Subscribe to [NestAudioFocusBus] for the lifetime of this VM (or
* until [teardown] cancels). On loss, mute every active listener
* + the broadcast mic without touching the user-visible mute
* states; on regain, restore the user's choice. Idempotent —
* a second call while the job is alive is a no-op.
*/
private fun ensureAudioFocusObserverStarted() {
if (audioFocusObserverJob?.isActive == true) return
audioFocusObserverJob =
viewModelScope.launch {
NestAudioFocusBus.state.collect { focusState ->
if (closed) return@collect
val newFocusMuted = focusState != NestAudioFocusState.Granted
if (newFocusMuted == focusMuted) return@collect
focusMuted = newFocusMuted
// Re-apply both directions. effectiveListenMuted()
// and the per-broadcast effective rebuild below
// both consume the just-updated [focusMuted] field.
applyEffectiveListenMute()
applyEffectiveMicMute()
}
}
}
/**
* Subscribe to [NestNetworkChangeBus] for the lifetime of this VM
* (or until [teardown]). On a default-network change, ask both
* the listener and the speaker wrappers to recycle their
* underlying sessions — much faster than waiting for the QUIC
* PTO on a dead socket. Idempotent.
*/
private fun ensureNetworkChangeObserverStarted() {
if (networkChangeObserverJob?.isActive == true) return
networkChangeObserverJob =
viewModelScope.launch {
NestNetworkChangeBus.events.collect {
if (closed) return@collect
// Best-effort. The wrapper-level recycleSession is a
// no-op on raw / non-reconnecting implementations
// (test fakes, IETF reference) so a network change
// there is silently ignored — the production paths
// wrap with [connectReconnectingNestsListener] /
// [connectReconnectingNestsSpeaker] which both
// override the hook to close the inner session.
listener?.runCatching { recycleSession() }
speaker?.runCatching { recycleSession() }
}
}
}
/**
* Push the effective mic mute (`user choice OR focus-loss`) onto
* the live broadcast handle, if any. The user-visible
* [BroadcastUiState.Broadcasting.isMuted] stays the user's choice
* — UI only reflects that, never the focus-driven temporary
* silencing — so a focus regain is invisible to the user.
*/
private fun applyEffectiveMicMute() {
val handle = broadcastHandle ?: return
val ui = _uiState.value.broadcast
val userMuted = (ui as? BroadcastUiState.Broadcasting)?.isMuted ?: false
val effective = userMuted || focusMuted
viewModelScope.launch {
runCatching { handle.setMuted(effective) }
}
}
fun setMuted(muted: Boolean) {
if (closed) return
_uiState.update { it.copy(isMuted = muted) }
activeSubscriptions.values.forEach { it.player?.setMutedSafe(muted) }
applyEffectiveListenMute()
}
/**
* Effective listener mute = user-chosen mute (from the talk-bar)
* OR audio-focus loss (phone call / nav prompt). Applied to every
* active player. The user-visible state (`NestUiState.isMuted`)
* stays the user's choice, so a focus regain restores it
* automatically.
*/
private fun effectiveListenMuted(): Boolean = _uiState.value.isMuted || focusMuted
private fun applyEffectiveListenMute() {
val effective = effectiveListenMuted()
activeSubscriptions.values.forEach { it.player?.setMutedSafe(effective) }
}
/**
@@ -523,7 +635,13 @@ class NestViewModel(
if (closed) return
val handle = broadcastHandle ?: return
viewModelScope.launch {
val result = handle.runCatching { setMuted(muted) }
// Apply (user-mute OR audio-focus-loss) to the wire so the
// mic stays silent through a phone call without us losing
// the user's intent. UI then reflects the user choice (not
// the focus-effective state) so a focus-driven mute doesn't
// visually flip the talk button.
val effective = muted || focusMuted
val result = handle.runCatching { setMuted(effective) }
if (closed) return@launch
_uiState.update {
val current = it.broadcast
@@ -866,7 +984,12 @@ class NestViewModel(
throw t
}
try {
val isMuted = _uiState.value.isMuted
// Apply (user-mute OR audio-focus-loss) so a speaker
// that comes on stage during a phone call attaches
// already silenced. The focus observer re-runs
// applyEffectiveListenMute() on regain, restoring the
// user's intent.
val isMuted = effectiveListenMuted()
val isHushed = pubkey in _uiState.value.locallyHushed
val roomPlayer =
NestPlayer(
@@ -940,6 +1063,20 @@ class NestViewModel(
announcesJob = null
levelEmitterJob?.cancel()
levelEmitterJob = null
// Audio-focus observation only ends on the final teardown —
// a transient disconnect+reconnect (user retry, room swap)
// keeps the bus subscription alive so a focus loss that
// happens during the gap is still picked up. The observer
// is idempotent under [ensureAudioFocusObserverStarted], so
// the next [connect] call is safe whether or not we
// cancelled here.
if (finalCleanup) {
audioFocusObserverJob?.cancel()
audioFocusObserverJob = null
focusMuted = false
networkChangeObserverJob?.cancel()
networkChangeObserverJob = null
}
rawAudioLevels.clear()
if (_audioLevels.value.isNotEmpty()) {
_audioLevels.value = emptyMap()
@@ -22,6 +22,10 @@ package com.vitorpamplona.nestsclient.audio
import android.media.AudioRecord
import android.media.MediaRecorder
import android.media.audiofx.AcousticEchoCanceler
import android.media.audiofx.AutomaticGainControl
import android.media.audiofx.NoiseSuppressor
import com.vitorpamplona.quartz.utils.Log
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import android.media.AudioFormat as AndroidAudioFormat
@@ -32,6 +36,19 @@ import android.media.AudioFormat as AndroidAudioFormat
* voice-chat libraries use, so it gets the platform's echo-cancellation and
* noise-suppression filters when available.
*
* **Audio effects (AEC / NS / AGC).** On most modern Android devices the
* `VOICE_COMMUNICATION` source is enough to engage the platform's echo
* canceller automatically. On a small set of older / OEM-customised
* devices it isn't — the AEC engine only attaches when the device is in
* `MODE_IN_COMMUNICATION`, which an audio-room app deliberately avoids
* driving (it reroutes everything through the call audio path and shows a
* "phone call" notification icon). To cover those devices without
* touching `AudioManager.mode` we explicitly attach the standalone
* [AcousticEchoCanceler] / [NoiseSuppressor] / [AutomaticGainControl]
* effects to the AudioRecord's session id. On devices where the source
* already engages them, attaching a second effect is a no-op (the
* platform deduplicates by session id) — so this is purely additive.
*
* **Permission:** the caller is responsible for holding `RECORD_AUDIO` before
* calling [start]; this class will throw [AudioException.Kind.DeviceUnavailable]
* if the OS denies the resource.
@@ -40,6 +57,9 @@ class AudioRecordCapture(
private val source: Int = MediaRecorder.AudioSource.VOICE_COMMUNICATION,
) : AudioCapture {
private var record: AudioRecord? = null
private var aec: AcousticEchoCanceler? = null
private var ns: NoiseSuppressor? = null
private var agc: AutomaticGainControl? = null
private var stopped = false
override fun start() {
@@ -97,6 +117,34 @@ class AudioRecordCapture(
)
}
record = rec
// Attach standalone audio effects to the session. This covers
// devices where the VOICE_COMMUNICATION source alone doesn't
// engage AEC (typically because they only attach AEC under
// MODE_IN_COMMUNICATION). All three are best-effort — a device
// that doesn't support an effect leaves it null without
// affecting capture.
attachAudioEffects(rec.audioSessionId)
}
private fun attachAudioEffects(sessionId: Int) {
if (AcousticEchoCanceler.isAvailable()) {
aec =
runCatching { AcousticEchoCanceler.create(sessionId)?.apply { enabled = true } }
.onFailure { Log.w("NestTx") { "AcousticEchoCanceler.create failed: ${it.message}" } }
.getOrNull()
}
if (NoiseSuppressor.isAvailable()) {
ns =
runCatching { NoiseSuppressor.create(sessionId)?.apply { enabled = true } }
.onFailure { Log.w("NestTx") { "NoiseSuppressor.create failed: ${it.message}" } }
.getOrNull()
}
if (AutomaticGainControl.isAvailable()) {
agc =
runCatching { AutomaticGainControl.create(sessionId)?.apply { enabled = true } }
.onFailure { Log.w("NestTx") { "AutomaticGainControl.create failed: ${it.message}" } }
.getOrNull()
}
}
override suspend fun readFrame(): ShortArray? {
@@ -128,6 +176,16 @@ class AudioRecordCapture(
override fun stop() {
if (stopped) return
stopped = true
// Release the audio effects BEFORE the AudioRecord — they hold
// a session-id reference and the platform expects effect
// teardown to precede the AudioRecord.release() that frees
// the session.
runCatching { aec?.release() }
runCatching { ns?.release() }
runCatching { agc?.release() }
aec = null
ns = null
agc = null
val rec = record ?: return
record = null
runCatching { rec.stop() }
@@ -116,6 +116,27 @@ interface NestsListener {
"announces() is moq-lite-only; IETF listener has no announce-prefix flow.",
)
/**
* Force the underlying transport / MoQ session to be torn down and
* a fresh one opened in its place — without permanently closing
* this listener. Used by the platform layer on a network change
* (Wi-Fi ↔ cellular handover) to skip the ~30 s QUIC PTO that
* would otherwise have to fire before the wrapper notices the
* old socket is dead.
*
* Default no-op for non-reconnecting implementations (raw
* [DefaultNestsListener] / [MoqLiteNestsListener]) — there's no
* orchestrator to drive a reconnect, so a force recycle would
* just close the listener. The reconnecting wrapper
* (`ReconnectingHandle` from [connectReconnectingNestsListener])
* overrides to close the active inner listener so its
* orchestrator opens a fresh session.
*/
suspend fun recycleSession() {
// no-op — only the reconnecting wrapper has anywhere to
// recycle to.
}
/** Tear down the MoQ session + underlying transport. Idempotent. */
suspend fun close()
}
@@ -66,6 +66,27 @@ interface NestsSpeaker {
*/
suspend fun startBroadcasting(onLevel: (Float) -> Unit = { /* no-op */ }): BroadcastHandle
/**
* Force the underlying transport / MoQ session to be torn down and
* a fresh one opened in its place — without permanently closing
* this speaker. Mirror of [NestsListener.recycleSession]; same
* use case (network handover) and same default no-op for non-
* reconnecting implementations.
*
* The reconnecting wrapper from
* [connectReconnectingNestsSpeaker] overrides to close the
* inner speaker so its orchestrator opens a fresh session.
* For moq-lite speakers, the long-lived
* [com.vitorpamplona.nestsclient.audio.NestMoqLiteBroadcaster]
* keeps capturing through the recycle and the
* [BroadcastHandle.swapPublisher]-based pump retargets onto
* the new session's publisher with no audible gap.
*/
suspend fun recycleSession() {
// no-op — only the reconnecting wrapper has anywhere to
// recycle to.
}
/** Tear down the MoQ session + transport. Idempotent. */
suspend fun close()
}
@@ -420,6 +420,29 @@ private class ReconnectingHandle(
)
}
/**
* Force-close the active inner listener so the orchestrator's
* `terminalAwait` returns Closed and the next loop iteration opens
* a fresh session. Used by the platform layer on a network change
* (Wi-Fi ↔ cellular) — without this, the wrapper would have to
* wait for the QUIC PTO to fire on the now-dead old socket
* (~30 s of silence) before noticing.
*
* The SubscribeHandle re-issuance pump cuts existing subs over to
* the new session as soon as it's Connected — same path the
* JWT-refresh recycle uses — so the consumer-facing
* [SubscribeHandle.objects] flow keeps emitting once the new
* session lands.
*/
override suspend fun recycleSession() {
val current = activeListener.value ?: return
// Best-effort. If the close races a concurrent reconnect path
// (extremely rare — we only call this on deliberate user /
// platform signals), the orchestrator absorbs both via its
// existing terminal-state handling.
runCatching { current.close() }
}
override suspend fun close() {
orchestrator.cancel()
runCatching { activeListener.value?.close() }
@@ -358,6 +358,17 @@ private class ReconnectingSpeakerHandle(
handle
}
/**
* Force-close the active inner speaker so the orchestrator opens
* a fresh session against the (presumably new) network. Used by
* the platform layer on a network change. Mirror of the listener
* wrapper's [com.vitorpamplona.nestsclient.NestsListener.recycleSession].
*/
override suspend fun recycleSession() {
val current = activeSpeaker.value ?: return
runCatching { current.close() }
}
override suspend fun close() {
orchestrator.cancel()
runCatching { activeBroadcast?.close() }