fix(nests): platform-side audio robustness — focus, AEC, route obs, network handover
Four follow-up fixes from the post-audit review (#4 / #5 / #6 / #7). #6 AcousticEchoCanceler / NoiseSuppressor / AGC on the AudioRecord session. The VOICE_COMMUNICATION input source engages the platform echo canceller automatically on most modern Android devices, but a small set of older / OEM-customised devices only attach AEC under MODE_IN_COMMUNICATION — which an audio-room app deliberately avoids. Attaching the standalone audiofx effects to the AudioRecord's session id covers those devices without rerouting through the call audio path. All three are best-effort and a no-op on devices where the source already engages them. #4 Real audio focus handling. The previous OnAudioFocusChangeListener was a no-op based on the assumption that the OS would auto-duck us; it doesn't (CONTENT_TYPE_SPEECH streams aren't auto-ducked). Inbound phone calls were mixing on top of room audio. - New `NestAudioFocusBus` (commons) — process-wide enum signal, decoupled from android.media so commons stays platform-free. - NestForegroundService translates AUDIOFOCUS_GAIN / LOSS_TRANSIENT* / LOSS into the bus enum. - NestViewModel observes the bus and silences both the listener playback (effective listen-mute = user OR focus) and the broadcast mic (effective mic-mute = user OR focus). User-visible mute states stay the user's choice so a focus regain restores them automatically. - The pipeline keeps running while focus is lost (decoder, capture, network) so unmute is sample-accurate when the call ends. #5 AudioDeviceCallback observability. Registers a callback in NestForegroundService that logs Bluetooth / wired / USB headset attach + detach with device type + name. Doesn't drive playback decisions — Android's auto-routing handles route swaps — but makes "audio cut out when I plugged in headphones" reports correlatable with a concrete event for the first time. #7 Network-change → fast reconnect. Without this, a Wi-Fi → cellular handover left the QUIC connection sitting on a now-dead socket until its PTO fired (~30 s) before the wrapper noticed. Now: - New `NestNetworkChangeBus` (commons) — collapses bursts of onLost/onAvailable into a single recycle event. - NestsListener + NestsSpeaker grow `recycleSession()` (default no-op); the reconnecting wrappers override to close the inner session so their orchestrator opens a fresh one. - NestForegroundService registers a default-network callback; suppresses the first onAvailable (registration callback) and only publishes on actual default-network changes. - NestViewModel observes the bus and calls recycleSession on both wrappers. The SubscribeHandle re-issuance pump (listener) and the hot-swap publisher pump (speaker) cut existing subscriptions / broadcasts onto the new session as soon as it lands — same paths the JWT-refresh recycle uses. - Manifest gains ACCESS_NETWORK_STATE for registerDefaultNetworkCallback.
This commit is contained in:
@@ -37,6 +37,11 @@
|
||||
<!-- To know receive notifications when the app connects/disconnects from the web -->
|
||||
<uses-permission android:name="android.permission.CHANGE_NETWORK_STATE" />
|
||||
|
||||
<!-- For ConnectivityManager.registerDefaultNetworkCallback used by NestForegroundService
|
||||
to detect Wi-Fi ↔ cellular handover and recycle the active QUIC session promptly
|
||||
(otherwise the wrapper waits ~30s for QUIC PTO before reconnecting). -->
|
||||
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
|
||||
|
||||
<!-- Audio/Video Playback -->
|
||||
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
|
||||
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK" />
|
||||
|
||||
+184
-12
@@ -28,13 +28,23 @@ import android.app.Service
|
||||
import android.content.Context
|
||||
import android.content.Intent
|
||||
import android.content.pm.ServiceInfo
|
||||
import android.media.AudioDeviceCallback
|
||||
import android.media.AudioDeviceInfo
|
||||
import android.media.AudioFocusRequest
|
||||
import android.media.AudioManager
|
||||
import android.net.ConnectivityManager
|
||||
import android.net.Network
|
||||
import android.os.Build
|
||||
import android.os.IBinder
|
||||
import android.os.PowerManager
|
||||
import androidx.core.app.NotificationCompat
|
||||
import androidx.core.content.ContextCompat
|
||||
import com.vitorpamplona.amethyst.R
|
||||
import com.vitorpamplona.amethyst.commons.viewmodels.NestAudioFocusBus
|
||||
import com.vitorpamplona.amethyst.commons.viewmodels.NestAudioFocusState
|
||||
import com.vitorpamplona.amethyst.commons.viewmodels.NestNetworkChangeBus
|
||||
import com.vitorpamplona.amethyst.ui.MainActivity
|
||||
import com.vitorpamplona.quartz.utils.Log
|
||||
|
||||
/**
|
||||
* Process-anchor for an active audio-room session. Holds a partial wake-lock
|
||||
@@ -57,7 +67,42 @@ import com.vitorpamplona.amethyst.ui.MainActivity
|
||||
class NestForegroundService : Service() {
|
||||
private var wakeLock: PowerManager.WakeLock? = null
|
||||
private var promoted = false
|
||||
private var audioFocusRequest: android.media.AudioFocusRequest? = null
|
||||
private var audioFocusRequest: AudioFocusRequest? = null
|
||||
|
||||
/**
|
||||
* Logs route changes (Bluetooth headset attach/detach, wired
|
||||
* headset, USB audio, speakerphone) so field reports of
|
||||
* "audio cut out when I plugged in headphones" can be correlated
|
||||
* with a concrete device-add / device-remove event.
|
||||
*
|
||||
* Doesn't drive playback decisions — Android's AudioTrack +
|
||||
* AudioRecord auto-route to whichever device the OS treats as
|
||||
* active, so the brief silence on a route swap is unavoidable
|
||||
* without going through the (intrusive) `MODE_IN_COMMUNICATION` +
|
||||
* `setCommunicationDevice` flow that this service deliberately
|
||||
* avoids. v1 ships observability only; future work could pause
|
||||
* playback briefly across a route change to mask the swap.
|
||||
*/
|
||||
private val deviceCallback =
|
||||
object : AudioDeviceCallback() {
|
||||
override fun onAudioDevicesAdded(addedDevices: Array<out AudioDeviceInfo>?) {
|
||||
addedDevices?.forEach { dev ->
|
||||
Log.i("NestAudio") {
|
||||
"audio device added: type=${dev.type} name='${dev.productName}' " +
|
||||
"isSink=${dev.isSink} isSource=${dev.isSource}"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
override fun onAudioDevicesRemoved(removedDevices: Array<out AudioDeviceInfo>?) {
|
||||
removedDevices?.forEach { dev ->
|
||||
Log.i("NestAudio") {
|
||||
"audio device removed: type=${dev.type} name='${dev.productName}' " +
|
||||
"isSink=${dev.isSink} isSource=${dev.isSource}"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
override fun onCreate() {
|
||||
super.onCreate()
|
||||
@@ -67,13 +112,103 @@ class NestForegroundService : Service() {
|
||||
.newWakeLock(PowerManager.PARTIAL_WAKE_LOCK, "amethyst:audio-room")
|
||||
.apply { setReferenceCounted(false) }
|
||||
requestAudioFocus()
|
||||
registerAudioDeviceCallback()
|
||||
registerNetworkCallback()
|
||||
}
|
||||
|
||||
private fun registerAudioDeviceCallback() {
|
||||
runCatching {
|
||||
val mgr = getSystemService(Context.AUDIO_SERVICE) as AudioManager
|
||||
// null Handler → callback runs on the main looper, which is
|
||||
// the cheapest path for "log a line" handlers like this.
|
||||
mgr.registerAudioDeviceCallback(deviceCallback, null)
|
||||
}
|
||||
}
|
||||
|
||||
private fun unregisterAudioDeviceCallback() {
|
||||
runCatching {
|
||||
val mgr = getSystemService(Context.AUDIO_SERVICE) as AudioManager
|
||||
mgr.unregisterAudioDeviceCallback(deviceCallback)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Request transient-may-duck audio focus so an inbound phone
|
||||
* call lowers the room volume cleanly instead of mixing two
|
||||
* voices on top of each other. Acquired once per service
|
||||
* lifetime; released in [onDestroy].
|
||||
* Track the current default network so the NetworkCallback can
|
||||
* distinguish "first onAvailable after register" (no-op) from "the
|
||||
* default network just changed under us" (publish). Volatile because
|
||||
* the callback fires on a binder thread; the publish path is
|
||||
* lock-free via [NestNetworkChangeBus].
|
||||
*/
|
||||
@Volatile private var currentDefaultNetwork: Network? = null
|
||||
|
||||
/**
|
||||
* Listens for the device's default-network changing (Wi-Fi ↔
|
||||
* cellular handover, plane mode toggle, hotspot swap) and signals
|
||||
* every active [com.vitorpamplona.amethyst.commons.viewmodels.NestViewModel]
|
||||
* to recycle its QUIC session. Without this nudge the QUIC
|
||||
* connection sitting on the now-dead socket would have to wait
|
||||
* for its PTO (~30 s) before the wrapper notices — a long
|
||||
* audible silence on every handover.
|
||||
*
|
||||
* The callback also fires once right after registration with the
|
||||
* current default network — we suppress that emission so the VM
|
||||
* doesn't recycle on every service start.
|
||||
*/
|
||||
private val networkCallback =
|
||||
object : ConnectivityManager.NetworkCallback() {
|
||||
override fun onAvailable(network: Network) {
|
||||
val previous = currentDefaultNetwork
|
||||
currentDefaultNetwork = network
|
||||
if (previous != null && previous != network) {
|
||||
Log.i("NestNet") {
|
||||
"default network changed ($previous → $network), recycling QUIC sessions"
|
||||
}
|
||||
NestNetworkChangeBus.publish()
|
||||
}
|
||||
}
|
||||
|
||||
override fun onLost(network: Network) {
|
||||
if (currentDefaultNetwork == network) {
|
||||
// We've lost the current default. Don't publish here —
|
||||
// the next onAvailable (with the replacement network)
|
||||
// will, and recycling NOW means the wrapper would try
|
||||
// to handshake on no network at all and fail-then-
|
||||
// backoff. Just clear so the next onAvailable is
|
||||
// recognised as a change.
|
||||
currentDefaultNetwork = null
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private fun registerNetworkCallback() {
|
||||
runCatching {
|
||||
val mgr = getSystemService(Context.CONNECTIVITY_SERVICE) as ConnectivityManager
|
||||
mgr.registerDefaultNetworkCallback(networkCallback)
|
||||
}
|
||||
}
|
||||
|
||||
private fun unregisterNetworkCallback() {
|
||||
runCatching {
|
||||
val mgr = getSystemService(Context.CONNECTIVITY_SERVICE) as ConnectivityManager
|
||||
mgr.unregisterNetworkCallback(networkCallback)
|
||||
}
|
||||
currentDefaultNetwork = null
|
||||
}
|
||||
|
||||
/**
|
||||
* Request audio focus for the duration of the audio-room session
|
||||
* and route the system's focus-change events into [NestAudioFocusBus]
|
||||
* so every active [com.vitorpamplona.amethyst.commons.viewmodels.NestViewModel]
|
||||
* can react.
|
||||
*
|
||||
* Why we actually handle the focus change (vs the previous no-op
|
||||
* listener): the platform only auto-ducks streams that opted into
|
||||
* auto-ducking (CONTENT_TYPE_MUSIC, etc.) — a `CONTENT_TYPE_SPEECH`
|
||||
* stream is left alone, so without a real listener an inbound phone
|
||||
* call would mix on top of the room audio and a Maps voice prompt
|
||||
* would be inaudible against an active speaker. The bus carries
|
||||
* the translated state to the VM, which silences the listener
|
||||
* playback and the broadcast mic for the duration of the loss.
|
||||
*
|
||||
* Matches the playback `AudioAttributes` we set on `AudioTrack`
|
||||
* in `AudioTrackPlayer` (USAGE_MEDIA + CONTENT_TYPE_SPEECH) so
|
||||
@@ -82,7 +217,7 @@ class NestForegroundService : Service() {
|
||||
*/
|
||||
private fun requestAudioFocus() {
|
||||
if (audioFocusRequest != null) return
|
||||
val mgr = getSystemService(Context.AUDIO_SERVICE) as android.media.AudioManager
|
||||
val mgr = getSystemService(Context.AUDIO_SERVICE) as AudioManager
|
||||
val attrs =
|
||||
android.media.AudioAttributes
|
||||
.Builder()
|
||||
@@ -90,24 +225,59 @@ class NestForegroundService : Service() {
|
||||
.setContentType(android.media.AudioAttributes.CONTENT_TYPE_SPEECH)
|
||||
.build()
|
||||
val request =
|
||||
android.media.AudioFocusRequest
|
||||
.Builder(android.media.AudioManager.AUDIOFOCUS_GAIN)
|
||||
AudioFocusRequest
|
||||
.Builder(AudioManager.AUDIOFOCUS_GAIN)
|
||||
.setAudioAttributes(attrs)
|
||||
.setAcceptsDelayedFocusGain(false)
|
||||
.setOnAudioFocusChangeListener({ /* no-op — duck-on-loss handled by the OS */ })
|
||||
.build()
|
||||
.setOnAudioFocusChangeListener { focusChange ->
|
||||
NestAudioFocusBus.publish(translateFocusChange(focusChange))
|
||||
}.build()
|
||||
// Best-effort: a refused request just means the OS will
|
||||
// duck/pause us based on its own policy. Don't fail the
|
||||
// service start over a focus denial.
|
||||
runCatching { mgr.requestAudioFocus(request) }
|
||||
val granted = runCatching { mgr.requestAudioFocus(request) }.getOrNull()
|
||||
// If the OS refused outright (rare — typically only when an
|
||||
// active call is already in progress at the moment we start),
|
||||
// publish TransientLoss immediately so the VM mutes from t=0
|
||||
// rather than playing for ~50 ms before the listener fires.
|
||||
// [AudioManager.AUDIOFOCUS_REQUEST_FAILED] = 0; granted = 1.
|
||||
if (granted == AudioManager.AUDIOFOCUS_REQUEST_FAILED) {
|
||||
NestAudioFocusBus.publish(NestAudioFocusState.TransientLoss)
|
||||
} else {
|
||||
NestAudioFocusBus.publish(NestAudioFocusState.Granted)
|
||||
}
|
||||
audioFocusRequest = request
|
||||
}
|
||||
|
||||
private fun abandonAudioFocus() {
|
||||
val request = audioFocusRequest ?: return
|
||||
val mgr = getSystemService(Context.AUDIO_SERVICE) as android.media.AudioManager
|
||||
val mgr = getSystemService(Context.AUDIO_SERVICE) as AudioManager
|
||||
runCatching { mgr.abandonAudioFocusRequest(request) }
|
||||
audioFocusRequest = null
|
||||
// Reset to Granted so a future foreground-service start that
|
||||
// happens before the next focus request lands doesn't inherit
|
||||
// a stale "muted because we lost focus" state.
|
||||
NestAudioFocusBus.publish(NestAudioFocusState.Granted)
|
||||
}
|
||||
|
||||
private fun translateFocusChange(focusChange: Int): NestAudioFocusState =
|
||||
when (focusChange) {
|
||||
AudioManager.AUDIOFOCUS_GAIN,
|
||||
AudioManager.AUDIOFOCUS_GAIN_TRANSIENT,
|
||||
AudioManager.AUDIOFOCUS_GAIN_TRANSIENT_MAY_DUCK,
|
||||
AudioManager.AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE,
|
||||
-> NestAudioFocusState.Granted
|
||||
|
||||
AudioManager.AUDIOFOCUS_LOSS_TRANSIENT,
|
||||
AudioManager.AUDIOFOCUS_LOSS_TRANSIENT_CAN_DUCK,
|
||||
-> NestAudioFocusState.TransientLoss
|
||||
|
||||
AudioManager.AUDIOFOCUS_LOSS -> NestAudioFocusState.Loss
|
||||
|
||||
// Unknown future codes: be defensive and treat as Granted
|
||||
// so a vendor-specific extension can't silently mute the
|
||||
// room forever.
|
||||
else -> NestAudioFocusState.Granted
|
||||
}
|
||||
|
||||
override fun onStartCommand(
|
||||
@@ -217,6 +387,8 @@ class NestForegroundService : Service() {
|
||||
wakeLock?.takeIf { it.isHeld }?.release()
|
||||
wakeLock = null
|
||||
abandonAudioFocus()
|
||||
unregisterAudioDeviceCallback()
|
||||
unregisterNetworkCallback()
|
||||
super.onDestroy()
|
||||
}
|
||||
|
||||
|
||||
+92
@@ -0,0 +1,92 @@
|
||||
/*
|
||||
* Copyright (c) 2025 Vitor Pamplona
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||
* this software and associated documentation files (the "Software"), to deal in
|
||||
* the Software without restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
|
||||
* Software, and to permit persons to whom the Software is furnished to do so,
|
||||
* subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in all
|
||||
* copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
|
||||
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
package com.vitorpamplona.amethyst.commons.viewmodels
|
||||
|
||||
import kotlinx.coroutines.flow.MutableStateFlow
|
||||
import kotlinx.coroutines.flow.StateFlow
|
||||
import kotlinx.coroutines.flow.asStateFlow
|
||||
|
||||
/**
|
||||
* Process-wide audio-focus signal published by the platform-side audio
|
||||
* focus listener (Android: `NestForegroundService`'s
|
||||
* `OnAudioFocusChangeListener`) and consumed by every active
|
||||
* [NestViewModel].
|
||||
*
|
||||
* Decoupled from the platform audio APIs so commons can stay free of
|
||||
* `android.media.AudioManager` references — the platform layer
|
||||
* translates focus-change codes into the small enum below before
|
||||
* publishing.
|
||||
*
|
||||
* Why a singleton: there's at most one foreground audio-room session
|
||||
* per process (the foreground service is unique), and every ViewModel
|
||||
* wants to observe the same focus signal, so keying by some other
|
||||
* dimension would just complicate the wiring. The bus survives
|
||||
* activity / VM rotations naturally.
|
||||
*/
|
||||
object NestAudioFocusBus {
|
||||
private val _state = MutableStateFlow(NestAudioFocusState.Granted)
|
||||
|
||||
/**
|
||||
* The latest audio-focus state. Defaults to [NestAudioFocusState.Granted]
|
||||
* — i.e. "we own playback" — so consumers don't have to special-case
|
||||
* the boot-up window before the foreground service has a chance to
|
||||
* register its listener.
|
||||
*/
|
||||
val state: StateFlow<NestAudioFocusState> = _state.asStateFlow()
|
||||
|
||||
/**
|
||||
* Publish a new focus state. Only the platform-side listener calls
|
||||
* this; consumers observe via [state].
|
||||
*/
|
||||
fun publish(newState: NestAudioFocusState) {
|
||||
_state.value = newState
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Audio-focus state translated from the platform's `AUDIOFOCUS_*` codes.
|
||||
* Maps them onto the three actions an audio-room app actually cares
|
||||
* about: keep playing, pause-because-something-else-is-playing, and
|
||||
* stop-because-something-else-is-now-the-primary-audio-source.
|
||||
*/
|
||||
enum class NestAudioFocusState {
|
||||
/** We own playback. Normal operation. */
|
||||
Granted,
|
||||
|
||||
/**
|
||||
* We've lost focus temporarily — typical triggers are an inbound
|
||||
* phone call, a maps voice prompt, a system alarm. The
|
||||
* [NestViewModel] reacts by silencing playback + the broadcast
|
||||
* mic (the user-visible "muted" state stays unchanged so it
|
||||
* restores on regain). Audio pipeline keeps running so resume
|
||||
* is sample-accurate.
|
||||
*/
|
||||
TransientLoss,
|
||||
|
||||
/**
|
||||
* Permanent focus loss — another app has taken over as the
|
||||
* primary audio source for the foreseeable future. Same effective
|
||||
* action as [TransientLoss] in v1 (silence both directions), but
|
||||
* the distinction is preserved so future enhancements can tear
|
||||
* down the audio device entirely on long-form loss.
|
||||
*/
|
||||
Loss,
|
||||
}
|
||||
+80
@@ -0,0 +1,80 @@
|
||||
/*
|
||||
* Copyright (c) 2025 Vitor Pamplona
|
||||
*
|
||||
* Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||
* this software and associated documentation files (the "Software"), to deal in
|
||||
* the Software without restriction, including without limitation the rights to use,
|
||||
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
|
||||
* Software, and to permit persons to whom the Software is furnished to do so,
|
||||
* subject to the following conditions:
|
||||
*
|
||||
* The above copyright notice and this permission notice shall be included in all
|
||||
* copies or substantial portions of the Software.
|
||||
*
|
||||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
|
||||
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
package com.vitorpamplona.amethyst.commons.viewmodels
|
||||
|
||||
import kotlinx.coroutines.channels.BufferOverflow
|
||||
import kotlinx.coroutines.flow.MutableSharedFlow
|
||||
import kotlinx.coroutines.flow.SharedFlow
|
||||
import kotlinx.coroutines.flow.asSharedFlow
|
||||
|
||||
/**
|
||||
* Process-wide signal published by the platform-side connectivity
|
||||
* listener (Android: `NestForegroundService`'s
|
||||
* `ConnectivityManager.NetworkCallback`) and consumed by every active
|
||||
* [NestViewModel] to recycle its [com.vitorpamplona.nestsclient.NestsListener]
|
||||
* / [com.vitorpamplona.nestsclient.NestsSpeaker] sessions.
|
||||
*
|
||||
* Why this exists: when a phone hands over from Wi-Fi to cellular (or
|
||||
* the other way), the local socket binding's source IP changes. The
|
||||
* QUIC connection sitting on the previous socket isn't notified of
|
||||
* the change — it'll keep retransmitting into the void until its PTO
|
||||
* fires (`~30 s` for an idle connection). Without an external nudge
|
||||
* the user hears 30 seconds of silence before the wrapper's reconnect
|
||||
* loop notices the failure. With this bus, the wrapper recycles the
|
||||
* QUIC session the moment the platform sees the network change,
|
||||
* shrinking the audible gap to a single re-handshake (≈ 1 s on
|
||||
* typical mobile networks).
|
||||
*
|
||||
* Decoupled from `android.net.ConnectivityManager` so commons stays
|
||||
* platform-free — the platform layer translates `onAvailable` /
|
||||
* `onLost` callbacks into a single `publish()` event before the
|
||||
* VM observes it.
|
||||
*/
|
||||
object NestNetworkChangeBus {
|
||||
/**
|
||||
* `extraBufferCapacity = 1, DROP_OLDEST` — the consumer only cares
|
||||
* that *a* network change happened, not how many. A burst of
|
||||
* onLost / onAvailable calls during a Wi-Fi flap collapses to a
|
||||
* single recycle event, which is exactly what we want (one
|
||||
* handshake instead of N).
|
||||
*/
|
||||
private val _events =
|
||||
MutableSharedFlow<Unit>(
|
||||
replay = 0,
|
||||
extraBufferCapacity = 1,
|
||||
onBufferOverflow = BufferOverflow.DROP_OLDEST,
|
||||
)
|
||||
|
||||
/**
|
||||
* Hot flow of network-change events. Consumers ([NestViewModel])
|
||||
* collect this and call `recycleSession()` on their listener +
|
||||
* speaker.
|
||||
*/
|
||||
val events: SharedFlow<Unit> = _events.asSharedFlow()
|
||||
|
||||
/**
|
||||
* Publish a network-change event. Only the platform-side listener
|
||||
* calls this; consumers observe via [events].
|
||||
*/
|
||||
fun publish() {
|
||||
_events.tryEmit(Unit)
|
||||
}
|
||||
}
|
||||
+140
-3
@@ -282,6 +282,34 @@ class NestViewModel(
|
||||
private var speakerStateJob: Job? = null
|
||||
private var speakerConnectJob: Job? = null
|
||||
|
||||
/**
|
||||
* Mirror of [NestAudioFocusBus.state] flipped to a boolean: `true`
|
||||
* while another app holds audio focus (inbound phone call, Maps
|
||||
* voice prompt, system alarm). Composed with the user-visible
|
||||
* mute states (`NestUiState.isMuted` for listeners,
|
||||
* `BroadcastUiState.Broadcasting.isMuted` for the speaker mic) via
|
||||
* [effectiveListenMuted] / [effectiveMicMuted] so the user-chosen
|
||||
* state is preserved across a focus loss + regain cycle.
|
||||
*
|
||||
* Maintained by [audioFocusObserverJob], started in [launchConnect]
|
||||
* (so a never-connected VM doesn't subscribe needlessly).
|
||||
*/
|
||||
@Volatile private var focusMuted: Boolean = false
|
||||
private var audioFocusObserverJob: Job? = null
|
||||
|
||||
/**
|
||||
* Subscribed in [ensureNetworkChangeObserverStarted]; calls
|
||||
* [com.vitorpamplona.nestsclient.NestsListener.recycleSession]
|
||||
* (and the speaker counterpart) when the platform reports a
|
||||
* default-network change. Without this, a Wi-Fi → cellular
|
||||
* handover leaves the QUIC connection sitting on the now-dead
|
||||
* socket until its PTO fires (~30 s) — the wrapper's reconnect
|
||||
* loop only learns about the failure at PTO. With the bus
|
||||
* nudge, recycle happens immediately and the audible gap
|
||||
* shrinks to a single re-handshake.
|
||||
*/
|
||||
private var networkChangeObserverJob: Job? = null
|
||||
|
||||
/**
|
||||
* Push the latest known speaker set from the room event. The user's
|
||||
* own pubkey (when broadcasting) is filtered out so we don't subscribe
|
||||
@@ -317,13 +345,97 @@ class NestViewModel(
|
||||
teardown(targetState = ConnectionUiState.Idle, finalCleanup = false)
|
||||
}
|
||||
|
||||
ensureAudioFocusObserverStarted()
|
||||
ensureNetworkChangeObserverStarted()
|
||||
launchConnect()
|
||||
}
|
||||
|
||||
/**
|
||||
* Subscribe to [NestAudioFocusBus] for the lifetime of this VM (or
|
||||
* until [teardown] cancels). On loss, mute every active listener
|
||||
* + the broadcast mic without touching the user-visible mute
|
||||
* states; on regain, restore the user's choice. Idempotent —
|
||||
* a second call while the job is alive is a no-op.
|
||||
*/
|
||||
private fun ensureAudioFocusObserverStarted() {
|
||||
if (audioFocusObserverJob?.isActive == true) return
|
||||
audioFocusObserverJob =
|
||||
viewModelScope.launch {
|
||||
NestAudioFocusBus.state.collect { focusState ->
|
||||
if (closed) return@collect
|
||||
val newFocusMuted = focusState != NestAudioFocusState.Granted
|
||||
if (newFocusMuted == focusMuted) return@collect
|
||||
focusMuted = newFocusMuted
|
||||
// Re-apply both directions. effectiveListenMuted()
|
||||
// and the per-broadcast effective rebuild below
|
||||
// both consume the just-updated [focusMuted] field.
|
||||
applyEffectiveListenMute()
|
||||
applyEffectiveMicMute()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Subscribe to [NestNetworkChangeBus] for the lifetime of this VM
|
||||
* (or until [teardown]). On a default-network change, ask both
|
||||
* the listener and the speaker wrappers to recycle their
|
||||
* underlying sessions — much faster than waiting for the QUIC
|
||||
* PTO on a dead socket. Idempotent.
|
||||
*/
|
||||
private fun ensureNetworkChangeObserverStarted() {
|
||||
if (networkChangeObserverJob?.isActive == true) return
|
||||
networkChangeObserverJob =
|
||||
viewModelScope.launch {
|
||||
NestNetworkChangeBus.events.collect {
|
||||
if (closed) return@collect
|
||||
// Best-effort. The wrapper-level recycleSession is a
|
||||
// no-op on raw / non-reconnecting implementations
|
||||
// (test fakes, IETF reference) so a network change
|
||||
// there is silently ignored — the production paths
|
||||
// wrap with [connectReconnectingNestsListener] /
|
||||
// [connectReconnectingNestsSpeaker] which both
|
||||
// override the hook to close the inner session.
|
||||
listener?.runCatching { recycleSession() }
|
||||
speaker?.runCatching { recycleSession() }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Push the effective mic mute (`user choice OR focus-loss`) onto
|
||||
* the live broadcast handle, if any. The user-visible
|
||||
* [BroadcastUiState.Broadcasting.isMuted] stays the user's choice
|
||||
* — UI only reflects that, never the focus-driven temporary
|
||||
* silencing — so a focus regain is invisible to the user.
|
||||
*/
|
||||
private fun applyEffectiveMicMute() {
|
||||
val handle = broadcastHandle ?: return
|
||||
val ui = _uiState.value.broadcast
|
||||
val userMuted = (ui as? BroadcastUiState.Broadcasting)?.isMuted ?: false
|
||||
val effective = userMuted || focusMuted
|
||||
viewModelScope.launch {
|
||||
runCatching { handle.setMuted(effective) }
|
||||
}
|
||||
}
|
||||
|
||||
fun setMuted(muted: Boolean) {
|
||||
if (closed) return
|
||||
_uiState.update { it.copy(isMuted = muted) }
|
||||
activeSubscriptions.values.forEach { it.player?.setMutedSafe(muted) }
|
||||
applyEffectiveListenMute()
|
||||
}
|
||||
|
||||
/**
|
||||
* Effective listener mute = user-chosen mute (from the talk-bar)
|
||||
* OR audio-focus loss (phone call / nav prompt). Applied to every
|
||||
* active player. The user-visible state (`NestUiState.isMuted`)
|
||||
* stays the user's choice, so a focus regain restores it
|
||||
* automatically.
|
||||
*/
|
||||
private fun effectiveListenMuted(): Boolean = _uiState.value.isMuted || focusMuted
|
||||
|
||||
private fun applyEffectiveListenMute() {
|
||||
val effective = effectiveListenMuted()
|
||||
activeSubscriptions.values.forEach { it.player?.setMutedSafe(effective) }
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -523,7 +635,13 @@ class NestViewModel(
|
||||
if (closed) return
|
||||
val handle = broadcastHandle ?: return
|
||||
viewModelScope.launch {
|
||||
val result = handle.runCatching { setMuted(muted) }
|
||||
// Apply (user-mute OR audio-focus-loss) to the wire so the
|
||||
// mic stays silent through a phone call without us losing
|
||||
// the user's intent. UI then reflects the user choice (not
|
||||
// the focus-effective state) so a focus-driven mute doesn't
|
||||
// visually flip the talk button.
|
||||
val effective = muted || focusMuted
|
||||
val result = handle.runCatching { setMuted(effective) }
|
||||
if (closed) return@launch
|
||||
_uiState.update {
|
||||
val current = it.broadcast
|
||||
@@ -866,7 +984,12 @@ class NestViewModel(
|
||||
throw t
|
||||
}
|
||||
try {
|
||||
val isMuted = _uiState.value.isMuted
|
||||
// Apply (user-mute OR audio-focus-loss) so a speaker
|
||||
// that comes on stage during a phone call attaches
|
||||
// already silenced. The focus observer re-runs
|
||||
// applyEffectiveListenMute() on regain, restoring the
|
||||
// user's intent.
|
||||
val isMuted = effectiveListenMuted()
|
||||
val isHushed = pubkey in _uiState.value.locallyHushed
|
||||
val roomPlayer =
|
||||
NestPlayer(
|
||||
@@ -940,6 +1063,20 @@ class NestViewModel(
|
||||
announcesJob = null
|
||||
levelEmitterJob?.cancel()
|
||||
levelEmitterJob = null
|
||||
// Audio-focus observation only ends on the final teardown —
|
||||
// a transient disconnect+reconnect (user retry, room swap)
|
||||
// keeps the bus subscription alive so a focus loss that
|
||||
// happens during the gap is still picked up. The observer
|
||||
// is idempotent under [ensureAudioFocusObserverStarted], so
|
||||
// the next [connect] call is safe whether or not we
|
||||
// cancelled here.
|
||||
if (finalCleanup) {
|
||||
audioFocusObserverJob?.cancel()
|
||||
audioFocusObserverJob = null
|
||||
focusMuted = false
|
||||
networkChangeObserverJob?.cancel()
|
||||
networkChangeObserverJob = null
|
||||
}
|
||||
rawAudioLevels.clear()
|
||||
if (_audioLevels.value.isNotEmpty()) {
|
||||
_audioLevels.value = emptyMap()
|
||||
|
||||
+58
@@ -22,6 +22,10 @@ package com.vitorpamplona.nestsclient.audio
|
||||
|
||||
import android.media.AudioRecord
|
||||
import android.media.MediaRecorder
|
||||
import android.media.audiofx.AcousticEchoCanceler
|
||||
import android.media.audiofx.AutomaticGainControl
|
||||
import android.media.audiofx.NoiseSuppressor
|
||||
import com.vitorpamplona.quartz.utils.Log
|
||||
import kotlinx.coroutines.Dispatchers
|
||||
import kotlinx.coroutines.withContext
|
||||
import android.media.AudioFormat as AndroidAudioFormat
|
||||
@@ -32,6 +36,19 @@ import android.media.AudioFormat as AndroidAudioFormat
|
||||
* voice-chat libraries use, so it gets the platform's echo-cancellation and
|
||||
* noise-suppression filters when available.
|
||||
*
|
||||
* **Audio effects (AEC / NS / AGC).** On most modern Android devices the
|
||||
* `VOICE_COMMUNICATION` source is enough to engage the platform's echo
|
||||
* canceller automatically. On a small set of older / OEM-customised
|
||||
* devices it isn't — the AEC engine only attaches when the device is in
|
||||
* `MODE_IN_COMMUNICATION`, which an audio-room app deliberately avoids
|
||||
* driving (it reroutes everything through the call audio path and shows a
|
||||
* "phone call" notification icon). To cover those devices without
|
||||
* touching `AudioManager.mode` we explicitly attach the standalone
|
||||
* [AcousticEchoCanceler] / [NoiseSuppressor] / [AutomaticGainControl]
|
||||
* effects to the AudioRecord's session id. On devices where the source
|
||||
* already engages them, attaching a second effect is a no-op (the
|
||||
* platform deduplicates by session id) — so this is purely additive.
|
||||
*
|
||||
* **Permission:** the caller is responsible for holding `RECORD_AUDIO` before
|
||||
* calling [start]; this class will throw [AudioException.Kind.DeviceUnavailable]
|
||||
* if the OS denies the resource.
|
||||
@@ -40,6 +57,9 @@ class AudioRecordCapture(
|
||||
private val source: Int = MediaRecorder.AudioSource.VOICE_COMMUNICATION,
|
||||
) : AudioCapture {
|
||||
private var record: AudioRecord? = null
|
||||
private var aec: AcousticEchoCanceler? = null
|
||||
private var ns: NoiseSuppressor? = null
|
||||
private var agc: AutomaticGainControl? = null
|
||||
private var stopped = false
|
||||
|
||||
override fun start() {
|
||||
@@ -97,6 +117,34 @@ class AudioRecordCapture(
|
||||
)
|
||||
}
|
||||
record = rec
|
||||
// Attach standalone audio effects to the session. This covers
|
||||
// devices where the VOICE_COMMUNICATION source alone doesn't
|
||||
// engage AEC (typically because they only attach AEC under
|
||||
// MODE_IN_COMMUNICATION). All three are best-effort — a device
|
||||
// that doesn't support an effect leaves it null without
|
||||
// affecting capture.
|
||||
attachAudioEffects(rec.audioSessionId)
|
||||
}
|
||||
|
||||
private fun attachAudioEffects(sessionId: Int) {
|
||||
if (AcousticEchoCanceler.isAvailable()) {
|
||||
aec =
|
||||
runCatching { AcousticEchoCanceler.create(sessionId)?.apply { enabled = true } }
|
||||
.onFailure { Log.w("NestTx") { "AcousticEchoCanceler.create failed: ${it.message}" } }
|
||||
.getOrNull()
|
||||
}
|
||||
if (NoiseSuppressor.isAvailable()) {
|
||||
ns =
|
||||
runCatching { NoiseSuppressor.create(sessionId)?.apply { enabled = true } }
|
||||
.onFailure { Log.w("NestTx") { "NoiseSuppressor.create failed: ${it.message}" } }
|
||||
.getOrNull()
|
||||
}
|
||||
if (AutomaticGainControl.isAvailable()) {
|
||||
agc =
|
||||
runCatching { AutomaticGainControl.create(sessionId)?.apply { enabled = true } }
|
||||
.onFailure { Log.w("NestTx") { "AutomaticGainControl.create failed: ${it.message}" } }
|
||||
.getOrNull()
|
||||
}
|
||||
}
|
||||
|
||||
override suspend fun readFrame(): ShortArray? {
|
||||
@@ -128,6 +176,16 @@ class AudioRecordCapture(
|
||||
override fun stop() {
|
||||
if (stopped) return
|
||||
stopped = true
|
||||
// Release the audio effects BEFORE the AudioRecord — they hold
|
||||
// a session-id reference and the platform expects effect
|
||||
// teardown to precede the AudioRecord.release() that frees
|
||||
// the session.
|
||||
runCatching { aec?.release() }
|
||||
runCatching { ns?.release() }
|
||||
runCatching { agc?.release() }
|
||||
aec = null
|
||||
ns = null
|
||||
agc = null
|
||||
val rec = record ?: return
|
||||
record = null
|
||||
runCatching { rec.stop() }
|
||||
|
||||
@@ -116,6 +116,27 @@ interface NestsListener {
|
||||
"announces() is moq-lite-only; IETF listener has no announce-prefix flow.",
|
||||
)
|
||||
|
||||
/**
|
||||
* Force the underlying transport / MoQ session to be torn down and
|
||||
* a fresh one opened in its place — without permanently closing
|
||||
* this listener. Used by the platform layer on a network change
|
||||
* (Wi-Fi ↔ cellular handover) to skip the ~30 s QUIC PTO that
|
||||
* would otherwise have to fire before the wrapper notices the
|
||||
* old socket is dead.
|
||||
*
|
||||
* Default no-op for non-reconnecting implementations (raw
|
||||
* [DefaultNestsListener] / [MoqLiteNestsListener]) — there's no
|
||||
* orchestrator to drive a reconnect, so a force recycle would
|
||||
* just close the listener. The reconnecting wrapper
|
||||
* (`ReconnectingHandle` from [connectReconnectingNestsListener])
|
||||
* overrides to close the active inner listener so its
|
||||
* orchestrator opens a fresh session.
|
||||
*/
|
||||
suspend fun recycleSession() {
|
||||
// no-op — only the reconnecting wrapper has anywhere to
|
||||
// recycle to.
|
||||
}
|
||||
|
||||
/** Tear down the MoQ session + underlying transport. Idempotent. */
|
||||
suspend fun close()
|
||||
}
|
||||
|
||||
@@ -66,6 +66,27 @@ interface NestsSpeaker {
|
||||
*/
|
||||
suspend fun startBroadcasting(onLevel: (Float) -> Unit = { /* no-op */ }): BroadcastHandle
|
||||
|
||||
/**
|
||||
* Force the underlying transport / MoQ session to be torn down and
|
||||
* a fresh one opened in its place — without permanently closing
|
||||
* this speaker. Mirror of [NestsListener.recycleSession]; same
|
||||
* use case (network handover) and same default no-op for non-
|
||||
* reconnecting implementations.
|
||||
*
|
||||
* The reconnecting wrapper from
|
||||
* [connectReconnectingNestsSpeaker] overrides to close the
|
||||
* inner speaker so its orchestrator opens a fresh session.
|
||||
* For moq-lite speakers, the long-lived
|
||||
* [com.vitorpamplona.nestsclient.audio.NestMoqLiteBroadcaster]
|
||||
* keeps capturing through the recycle and the
|
||||
* [BroadcastHandle.swapPublisher]-based pump retargets onto
|
||||
* the new session's publisher with no audible gap.
|
||||
*/
|
||||
suspend fun recycleSession() {
|
||||
// no-op — only the reconnecting wrapper has anywhere to
|
||||
// recycle to.
|
||||
}
|
||||
|
||||
/** Tear down the MoQ session + transport. Idempotent. */
|
||||
suspend fun close()
|
||||
}
|
||||
|
||||
+23
@@ -420,6 +420,29 @@ private class ReconnectingHandle(
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Force-close the active inner listener so the orchestrator's
|
||||
* `terminalAwait` returns Closed and the next loop iteration opens
|
||||
* a fresh session. Used by the platform layer on a network change
|
||||
* (Wi-Fi ↔ cellular) — without this, the wrapper would have to
|
||||
* wait for the QUIC PTO to fire on the now-dead old socket
|
||||
* (~30 s of silence) before noticing.
|
||||
*
|
||||
* The SubscribeHandle re-issuance pump cuts existing subs over to
|
||||
* the new session as soon as it's Connected — same path the
|
||||
* JWT-refresh recycle uses — so the consumer-facing
|
||||
* [SubscribeHandle.objects] flow keeps emitting once the new
|
||||
* session lands.
|
||||
*/
|
||||
override suspend fun recycleSession() {
|
||||
val current = activeListener.value ?: return
|
||||
// Best-effort. If the close races a concurrent reconnect path
|
||||
// (extremely rare — we only call this on deliberate user /
|
||||
// platform signals), the orchestrator absorbs both via its
|
||||
// existing terminal-state handling.
|
||||
runCatching { current.close() }
|
||||
}
|
||||
|
||||
override suspend fun close() {
|
||||
orchestrator.cancel()
|
||||
runCatching { activeListener.value?.close() }
|
||||
|
||||
+11
@@ -358,6 +358,17 @@ private class ReconnectingSpeakerHandle(
|
||||
handle
|
||||
}
|
||||
|
||||
/**
|
||||
* Force-close the active inner speaker so the orchestrator opens
|
||||
* a fresh session against the (presumably new) network. Used by
|
||||
* the platform layer on a network change. Mirror of the listener
|
||||
* wrapper's [com.vitorpamplona.nestsclient.NestsListener.recycleSession].
|
||||
*/
|
||||
override suspend fun recycleSession() {
|
||||
val current = activeSpeaker.value ?: return
|
||||
runCatching { current.close() }
|
||||
}
|
||||
|
||||
override suspend fun close() {
|
||||
orchestrator.cancel()
|
||||
runCatching { activeBroadcast?.close() }
|
||||
|
||||
Reference in New Issue
Block a user