feat(audio-rooms): M8 polish + M3/M9 foreground service

M8a — presence event reflects mic-mute state:
- AudioRoomStage's `publishPresence` now passes the broadcaster's
  current mic state into the kind 10312 `muted` tag: `null` when not
  broadcasting (no mic to be muted on), explicit `true` / `false` while
  the speaker path is `Broadcasting`. Other clients can now render a
  mute indicator on our avatar.

M8b — auto-reconnect with capped exponential backoff:
- AudioRoomViewModel detects `NestsListenerState.Failed` and schedules a
  retry after 1s, 2s, 4s, ..., capped at 16s. Up to 3 attempts before
  the UI stays in Failed for a manual retry.
- User-initiated `connect()` and `disconnect()` reset the retry counter
  so manual recovery starts fresh.

M8c — iOS:
- Skipped: neither `:nestsClient` nor `:quic` declares an iOS target
  yet, so there's no iosMain source set to populate. When iOS lands,
  the audio capture/playback + transport actuals will need iOS impls
  (and the speaker UI will need an iOS shell).

M3 + M9 — foreground service:
- New `AudioRoomForegroundService` (foregroundServiceType
  `mediaPlayback|microphone`) anchors the process so audio keeps
  playing with the screen off. Holds a partial wake-lock + media-style
  notification; the notification's "Stop" action stops the service.
- Lifecycle wired in AudioRoomStage via:
  * LaunchedEffect(isConnected, isBroadcasting) on the listener +
    broadcast UI state — promotes to mediaPlayback+microphone type
    when broadcasting starts (Android 14+ split foreground-type
    permission requirement), falls back to mediaPlayback when only
    listening, stops entirely when listener drops.
  * DisposableEffect(Unit) for screen-exit cleanup.
- The service does NOT own the MoQ session / decoder / player — those
  remain in the VM. Screen-off works; "navigate away keeps audio" would
  require moving the audio stack into the service, which is a bigger
  refactor outside the audio-rooms completion plan's scope.
- Strings: 6 new `audio_room_notification_*` keys.
- Manifest: declares the service with `mediaPlayback|microphone`
  foreground type. RECORD_AUDIO + FOREGROUND_SERVICE_MICROPHONE were
  already declared.

Verified: `./gradlew spotlessApply :commons:jvmTest :nestsClient:jvmTest
:quic:jvmTest :amethyst:compilePlayDebugKotlin` all green.
This commit is contained in:
Claude
2026-04-26 03:34:28 +00:00
parent 1c6d939d28
commit 0a45d1094f
5 changed files with 382 additions and 38 deletions
+6
View File
@@ -260,6 +260,12 @@
android:stopWithTask="false"
android:exported="false" />
<service
android:name=".service.audiorooms.AudioRoomForegroundService"
android:foregroundServiceType="mediaPlayback|microphone"
android:stopWithTask="true"
android:exported="false" />
<provider
android:name="androidx.core.content.FileProvider"
android:authorities="${applicationId}.provider"
@@ -0,0 +1,205 @@
/*
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.amethyst.service.audiorooms
import android.app.Notification
import android.app.NotificationChannel
import android.app.NotificationManager
import android.app.PendingIntent
import android.app.Service
import android.content.Context
import android.content.Intent
import android.content.pm.ServiceInfo
import android.os.Build
import android.os.IBinder
import android.os.PowerManager
import androidx.core.app.NotificationCompat
import androidx.core.content.ContextCompat
import com.vitorpamplona.amethyst.R
import com.vitorpamplona.amethyst.ui.MainActivity
/**
* Process-anchor for an active audio-room session. Holds a partial wake-lock
* and a foreground notification so playback continues when the screen is off
* or the user briefly leaves the app.
*
* Scope decisions:
* - The service does NOT own the MoQ session / decoder / player. Those
* live in `AudioRoomViewModel`. This service exists to keep the process
* alive while audio is in flight; the VM still drives all wire activity.
* - Foreground type is `mediaPlayback`; if the user starts broadcasting,
* the screen calls [promoteToMicrophone] which re-`startForeground`s
* with `mediaPlayback|microphone` (Android 14+ requires the explicit
* microphone type while the mic is open).
* - One service per process — multiple audio-room screens are not a real
* scenario today (there's one room per screen and the room screen is a
* full-screen activity). The service uses a shared notification id so
* start/promote/stop is idempotent.
*/
class AudioRoomForegroundService : Service() {
private var wakeLock: PowerManager.WakeLock? = null
private var promoted = false
override fun onCreate() {
super.onCreate()
createNotificationChannel()
wakeLock =
(getSystemService(Context.POWER_SERVICE) as PowerManager)
.newWakeLock(PowerManager.PARTIAL_WAKE_LOCK, "amethyst:audio-room")
.apply { setReferenceCounted(false) }
}
override fun onStartCommand(
intent: Intent?,
flags: Int,
startId: Int,
): Int {
when (intent?.action) {
ACTION_PROMOTE_TO_MIC -> {
startForegroundWithType(includeMic = true)
}
ACTION_STOP -> {
stopForeground(STOP_FOREGROUND_REMOVE)
stopSelf()
}
else -> {
startForegroundWithType(includeMic = promoted)
}
}
if (wakeLock?.isHeld != true) {
wakeLock?.acquire(WAKE_LOCK_TIMEOUT_MS)
}
return START_STICKY
}
private fun startForegroundWithType(includeMic: Boolean) {
promoted = includeMic
val notification = buildNotification()
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.UPSIDE_DOWN_CAKE) {
val type =
if (includeMic) {
ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PLAYBACK or
ServiceInfo.FOREGROUND_SERVICE_TYPE_MICROPHONE
} else {
ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PLAYBACK
}
startForeground(NOTIFICATION_ID, notification, type)
} else {
startForeground(NOTIFICATION_ID, notification)
}
}
private fun buildNotification(): Notification {
val openIntent =
PendingIntent.getActivity(
this,
0,
Intent(this, MainActivity::class.java),
PendingIntent.FLAG_IMMUTABLE or PendingIntent.FLAG_UPDATE_CURRENT,
)
val stopIntent =
PendingIntent.getService(
this,
1,
Intent(this, AudioRoomForegroundService::class.java).apply { action = ACTION_STOP },
PendingIntent.FLAG_IMMUTABLE or PendingIntent.FLAG_UPDATE_CURRENT,
)
val title =
if (promoted) {
getString(R.string.audio_room_notification_broadcasting)
} else {
getString(R.string.audio_room_notification_listening)
}
return NotificationCompat
.Builder(this, CHANNEL_ID)
.setContentTitle(title)
.setContentText(getString(R.string.audio_room_notification_text))
.setSmallIcon(R.drawable.amethyst)
.setOngoing(true)
.setContentIntent(openIntent)
.addAction(0, getString(R.string.audio_room_notification_stop), stopIntent)
.setCategory(NotificationCompat.CATEGORY_CALL)
.build()
}
private fun createNotificationChannel() {
val mgr = getSystemService(Context.NOTIFICATION_SERVICE) as NotificationManager
if (mgr.getNotificationChannel(CHANNEL_ID) == null) {
mgr.createNotificationChannel(
NotificationChannel(
CHANNEL_ID,
getString(R.string.audio_room_notification_channel),
NotificationManager.IMPORTANCE_LOW,
).apply {
description = getString(R.string.audio_room_notification_channel_description)
setShowBadge(false)
},
)
}
}
override fun onDestroy() {
wakeLock?.takeIf { it.isHeld }?.release()
wakeLock = null
super.onDestroy()
}
override fun onBind(intent: Intent?): IBinder? = null
companion object {
private const val CHANNEL_ID = "audio_room_foreground"
private const val NOTIFICATION_ID = 0xA0D10
private const val ACTION_PROMOTE_TO_MIC = "com.vitorpamplona.amethyst.audio_room.PROMOTE_MIC"
private const val ACTION_STOP = "com.vitorpamplona.amethyst.audio_room.STOP"
private const val WAKE_LOCK_TIMEOUT_MS = 12L * 60 * 60 * 1000 // 12 hours hard cap
/** Start the service in listener-only foreground mode. Idempotent. */
fun startListening(context: Context) {
ContextCompat.startForegroundService(
context,
Intent(context, AudioRoomForegroundService::class.java),
)
}
/**
* Re-`startForeground` with mediaPlayback + microphone type while the
* user is broadcasting. Required by Android 14's split foreground-type
* permission model.
*/
fun promoteToMicrophone(context: Context) {
ContextCompat.startForegroundService(
context,
Intent(context, AudioRoomForegroundService::class.java).apply {
action = ACTION_PROMOTE_TO_MIC
},
)
}
/** Stop and remove the foreground notification. Idempotent. */
fun stop(context: Context) {
context.stopService(Intent(context, AudioRoomForegroundService::class.java))
}
}
}
@@ -68,6 +68,7 @@ import com.vitorpamplona.amethyst.commons.model.nip53LiveActivities.LiveActiviti
import com.vitorpamplona.amethyst.commons.viewmodels.AudioRoomViewModel
import com.vitorpamplona.amethyst.commons.viewmodels.BroadcastUiState
import com.vitorpamplona.amethyst.commons.viewmodels.ConnectionUiState
import com.vitorpamplona.amethyst.service.audiorooms.AudioRoomForegroundService
import com.vitorpamplona.amethyst.ui.note.ClickableUserPicture
import com.vitorpamplona.amethyst.ui.note.LoadAddressableNote
import com.vitorpamplona.amethyst.ui.screen.loggedIn.AccountViewModel
@@ -95,20 +96,19 @@ import kotlinx.coroutines.launch
* Clubhouse-style audio-room "stage" rendered in place of the video player when
* the underlying activity is a NIP-53 kind 30312 [MeetingSpaceEvent].
*
* Responsibilities (M1 = listener-only):
* Responsibilities:
* - Displays host / speaker / audience avatars parsed from the 30312 `p` tags.
* - Publishes kind 10312 presence on enter and every 30 s while composed.
* - Hand-raise toggle flips the `["hand","1"|"0"]` tag on that presence event
* so a host on any NIP-53 client can see the request and promote.
* - **Connect button** opens the listener-side audio pipeline (HTTP →
* WebTransport over QUIC → MoQ → Opus decode → AudioTrack). On Connected,
* auto-subscribes to every host's speaker track.
* - **Mute toggle** silences the local audio device without halting the
* network pipeline so unmute is instant.
*
* Speaker-side (mic capture + publish) is M5+ in the audio-rooms completion
* plan — `nestsClient/plans/2026-04-26-audio-rooms-completion.md`. Until then
* the presence event omits the `muted` tag (we have no mic to mute).
* Avatars get a primary-color ring while their MoQ track is delivering audio.
* - Publishes kind 10312 presence on enter and every 30 s while composed,
* reflecting the hand-raise + mic-mute state.
* - **Listener** path: Connect button → HTTP → WebTransport over QUIC → MoQ
* listener session → Opus decode → AudioTrack. Auto-subscribes to every
* host + speaker track. Mute toggle silences the local device without
* halting the network so unmute is instant.
* - **Speaker** path (only for users in the room's `p` tags as host or
* speaker): Talk button → RECORD_AUDIO permission → second MoQ session
* in publisher mode → AudioRecord → MediaCodec Opus encoder → MoQ
* OBJECT_DATAGRAM emission. Live indicator + mic-mute toggle.
*/
@Composable
fun AudioRoomStage(
@@ -144,25 +144,6 @@ private fun AudioRoomStageContent(
val scope = rememberCoroutineScope()
val account = accountViewModel.account
// Publish initial presence on enter and refresh every PRESENCE_REFRESH_MS while composed.
LaunchedEffect(event.address().toValue(), handRaised) {
publishPresence(account, event, handRaised)
while (isActive) {
delay(PRESENCE_REFRESH_MS)
publishPresence(account, event, handRaised)
}
}
// Best-effort "leave" — re-publish a lowered-hand presence so peers see us
// drop sooner than the 30 s heartbeat would otherwise allow.
DisposableEffect(event.address().toValue()) {
onDispose {
scope.launch(Dispatchers.IO) {
runCatching { publishPresence(account, event, handRaised = false) }
}
}
}
val serviceBase = event.service()
val roomId = event.address().dTag
val audioAvailable = !serviceBase.isNullOrBlank() && roomId.isNotBlank()
@@ -193,6 +174,52 @@ private fun AudioRoomStageContent(
val ui = viewModel?.uiState?.collectAsState()?.value
val speakingNow = ui?.speakingNow ?: persistentSetOf()
// Foreground-service lifecycle. The service is a process-anchor — it
// doesn't own the audio resources (those live in the VM) but its
// foreground notification + wake-lock keep audio playing with the
// screen off. Promotes to mediaPlayback+microphone type while the user
// broadcasts (Android 14+ split foreground-type permission model).
val context = LocalContext.current
val isConnected = ui?.connection is ConnectionUiState.Connected
val isBroadcasting = ui?.broadcast is BroadcastUiState.Broadcasting
LaunchedEffect(isConnected, isBroadcasting) {
when {
isConnected && isBroadcasting -> AudioRoomForegroundService.promoteToMicrophone(context)
isConnected -> AudioRoomForegroundService.startListening(context)
else -> AudioRoomForegroundService.stop(context)
}
}
DisposableEffect(Unit) {
onDispose { AudioRoomForegroundService.stop(context) }
}
// Mic state for the presence event: null when not broadcasting (we have
// no mic stream to be muted on), explicit true/false while live so other
// clients can render the mute indicator on our avatar.
val micMutedTag: Boolean? =
when (val b = ui?.broadcast) {
is BroadcastUiState.Broadcasting -> b.isMuted
else -> null
}
// Publish initial presence on enter and refresh every PRESENCE_REFRESH_MS while composed.
LaunchedEffect(event.address().toValue(), handRaised, micMutedTag) {
publishPresence(account, event, handRaised, micMutedTag)
while (isActive) {
delay(PRESENCE_REFRESH_MS)
publishPresence(account, event, handRaised, micMutedTag)
}
}
// Best-effort "leave" — re-publish a lowered-hand presence so peers see us
// drop sooner than the 30 s heartbeat would otherwise allow.
DisposableEffect(event.address().toValue()) {
onDispose {
scope.launch(Dispatchers.IO) {
runCatching { publishPresence(account, event, handRaised = false, micMuted = null) }
}
}
}
Card(
modifier = Modifier.fillMaxWidth().padding(8.dp),
shape = RoundedCornerShape(12.dp),
@@ -530,16 +557,17 @@ private suspend fun publishPresence(
account: com.vitorpamplona.amethyst.model.Account,
event: MeetingSpaceEvent,
handRaised: Boolean,
micMuted: Boolean?,
) {
runCatching {
account.signAndComputeBroadcast(
MeetingRoomPresenceEvent.build(
root = event,
handRaised = handRaised,
// muted tag intentionally omitted — listener-only mute (M1)
// silences our speakers, not a mic we don't yet broadcast.
// Re-evaluate when M5 (publisher path) lands.
muted = null,
// null = not broadcasting (no mic stream); true/false reflect
// the speaker's current mic state so other clients can show a
// mute indicator on the avatar.
muted = micMuted,
),
)
}
+6
View File
@@ -512,6 +512,12 @@
<string name="audio_room_broadcasting">Live</string>
<string name="audio_room_broadcast_failed">Broadcast failed: %1$s</string>
<string name="audio_room_record_permission_required">Microphone access is required to talk in this room.</string>
<string name="audio_room_notification_channel">Audio rooms</string>
<string name="audio_room_notification_channel_description">Keeps audio playing while a room is open.</string>
<string name="audio_room_notification_listening">Audio room connected</string>
<string name="audio_room_notification_broadcasting">Audio room — Live</string>
<string name="audio_room_notification_text">Tap to return.</string>
<string name="audio_room_notification_stop">Stop</string>
<string name="longs">Videos</string>
<string name="articles">Articles</string>
<string name="private_bookmarks">Private Bookmarks</string>
@@ -109,6 +109,12 @@ class AudioRoomViewModel(
private var requestedSpeakers: Set<String> = emptySet()
private var closed = false
// Auto-reconnect state — incremented every time we observe Failed and
// schedule a retry; reset to 0 on Connected or on a user-initiated
// connect()/disconnect() so manual recovery starts fresh.
private var autoRetryAttempts = 0
private var autoRetryJob: Job? = null
// Speaker / publisher path
private var speaker: NestsSpeaker? = null
private var broadcastHandle: BroadcastHandle? = null
@@ -133,6 +139,11 @@ class AudioRoomViewModel(
val current = _uiState.value.connection
if (current is ConnectionUiState.Connecting || current is ConnectionUiState.Connected) return
// User-initiated connect — clear any pending auto-retry timer + counter.
autoRetryJob?.cancel()
autoRetryJob = null
autoRetryAttempts = 0
_uiState.update { it.copy(connection = ConnectionUiState.Connecting(ConnectionUiState.Step.ResolvingRoom)) }
connectJob =
@@ -264,12 +275,17 @@ class AudioRoomViewModel(
/** Tear down without finalizing the VM (e.g. user pressed Disconnect). */
fun disconnect() {
if (closed) return
autoRetryJob?.cancel()
autoRetryJob = null
autoRetryAttempts = 0
teardownBroadcast(BroadcastUiState.Idle)
teardown(targetState = ConnectionUiState.Idle)
}
override fun onCleared() {
closed = true
autoRetryJob?.cancel()
autoRetryJob = null
teardownBroadcast(BroadcastUiState.Idle)
teardown(targetState = ConnectionUiState.Closed)
super.onCleared()
@@ -323,13 +339,87 @@ class AudioRoomViewModel(
_uiState.update { ui ->
ui.copy(connection = state.toUiState(ui.connection))
}
if (state is NestsListenerState.Connected) {
reconcileSubscriptions()
when (state) {
is NestsListenerState.Connected -> {
autoRetryAttempts = 0
reconcileSubscriptions()
}
is NestsListenerState.Failed -> {
scheduleAutoRetry()
}
else -> { /* no extra side effect */ }
}
}
}
}
/**
* Auto-reconnect on listener Failed with capped exponential backoff.
* The user can still tap Connect manually at any time; that resets the
* retry counter via the `connect()` path.
*/
private fun scheduleAutoRetry() {
if (closed) return
if (autoRetryJob?.isActive == true) return
if (autoRetryAttempts >= MAX_AUTO_RETRIES) return
val attempt = autoRetryAttempts
autoRetryAttempts = attempt + 1
val backoffMs = minOf(MAX_RETRY_BACKOFF_MS, INITIAL_RETRY_BACKOFF_MS shl attempt)
autoRetryJob =
viewModelScope.launch {
delay(backoffMs)
if (closed) return@launch
if (_uiState.value.connection !is ConnectionUiState.Failed) return@launch
// Drop the previous listener cleanly before starting a new
// attempt. Reset state to Idle so the connect() guard passes.
teardown(targetState = ConnectionUiState.Idle)
connectInternal()
}
}
/**
* Internal connect that bypasses the auto-retry counter reset used by
* [scheduleAutoRetry] so successive auto-retries continue to back off.
*/
private fun connectInternal() {
if (closed) return
val current = _uiState.value.connection
if (current is ConnectionUiState.Connecting || current is ConnectionUiState.Connected) return
_uiState.update { it.copy(connection = ConnectionUiState.Connecting(ConnectionUiState.Step.ResolvingRoom)) }
connectJob =
viewModelScope.launch {
try {
val l =
connector.connect(
httpClient = httpClient,
transport = transport,
scope = viewModelScope,
serviceBase = serviceBase,
roomId = roomId,
signer = signer,
)
if (closed) {
runCatching { l.close() }
return@launch
}
listener = l
observeListenerState(l)
} catch (ce: CancellationException) {
throw ce
} catch (t: Throwable) {
_uiState.update {
it.copy(connection = ConnectionUiState.Failed(t.message ?: t::class.simpleName ?: "connect failed"))
}
scheduleAutoRetry()
}
}
}
private fun reconcileSubscriptions() {
val l = listener ?: return
if (_uiState.value.connection !is ConnectionUiState.Connected) return
@@ -588,6 +678,15 @@ sealed class BroadcastUiState {
*/
const val SPEAKING_TIMEOUT_MS: Long = 250L
/** Max number of auto-reconnect attempts after a Failed listener state. */
private const val MAX_AUTO_RETRIES = 3
/** First auto-retry backoff in ms; doubles each subsequent attempt. */
private const val INITIAL_RETRY_BACKOFF_MS = 1_000L
/** Cap on the auto-retry backoff so the wait stays human-acceptable. */
private const val MAX_RETRY_BACKOFF_MS = 16_000L
/**
* Indirection over the top-level `connectNestsListener` so tests can drive
* a fake [NestsListener] directly without standing up an HTTP fake +