feat(audio-rooms): M8 polish + M3/M9 foreground service

M8a — presence event reflects mic-mute state: - AudioRoomStage's `publishPresence` now passes the broadcaster's current mic state into the kind 10312 `muted` tag: `null` when not broadcasting (no mic to be muted on), explicit `true` / `false` while the speaker path is `Broadcasting`. Other clients can now render a mute indicator on our avatar. M8b — auto-reconnect with capped exponential backoff: - AudioRoomViewModel detects `NestsListenerState.Failed` and schedules a retry after 1s, 2s, 4s, ..., capped at 16s. Up to 3 attempts before the UI stays in Failed for a manual retry. - User-initiated `connect()` and `disconnect()` reset the retry counter so manual recovery starts fresh. M8c — iOS: - Skipped: neither `:nestsClient` nor `:quic` declares an iOS target yet, so there's no iosMain source set to populate. When iOS lands, the audio capture/playback + transport actuals will need iOS impls (and the speaker UI will need an iOS shell). M3 + M9 — foreground service: - New `AudioRoomForegroundService` (foregroundServiceType `mediaPlayback|microphone`) anchors the process so audio keeps playing with the screen off. Holds a partial wake-lock + media-style notification; the notification's "Stop" action stops the service. - Lifecycle wired in AudioRoomStage via: * LaunchedEffect(isConnected, isBroadcasting) on the listener + broadcast UI state — promotes to mediaPlayback+microphone type when broadcasting starts (Android 14+ split foreground-type permission requirement), falls back to mediaPlayback when only listening, stops entirely when listener drops. * DisposableEffect(Unit) for screen-exit cleanup. - The service does NOT own the MoQ session / decoder / player — those remain in the VM. Screen-off works; "navigate away keeps audio" would require moving the audio stack into the service, which is a bigger refactor outside the audio-rooms completion plan's scope. - Strings: 6 new `audio_room_notification_*` keys. - Manifest: declares the service with `mediaPlayback|microphone` foreground type. RECORD_AUDIO + FOREGROUND_SERVICE_MICROPHONE were already declared. Verified: `./gradlew spotlessApply :commons:jvmTest :nestsClient:jvmTest :quic:jvmTest :amethyst:compilePlayDebugKotlin` all green.
2026-04-26 03:34:28 +00:00
parent 1c6d939d28
commit 0a45d1094f
5 changed files with 382 additions and 38 deletions
@@ -260,6 +260,12 @@
            android:stopWithTask="false"
            android:exported="false" />

+        <service
+            android:name=".service.audiorooms.AudioRoomForegroundService"
+            android:foregroundServiceType="mediaPlayback|microphone"
+            android:stopWithTask="true"
+            android:exported="false" />
+
        <provider
            android:name="androidx.core.content.FileProvider"
            android:authorities="${applicationId}.provider"
@@ -0,0 +1,205 @@
+/*
+ * Copyright (c) 2025 Vitor Pamplona
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy of
+ * this software and associated documentation files (the "Software"), to deal in
+ * the Software without restriction, including without limitation the rights to use,
+ * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
+ * Software, and to permit persons to whom the Software is furnished to do so,
+ * subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in all
+ * copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+ * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+ * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
+ * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+package com.vitorpamplona.amethyst.service.audiorooms
+
+import android.app.Notification
+import android.app.NotificationChannel
+import android.app.NotificationManager
+import android.app.PendingIntent
+import android.app.Service
+import android.content.Context
+import android.content.Intent
+import android.content.pm.ServiceInfo
+import android.os.Build
+import android.os.IBinder
+import android.os.PowerManager
+import androidx.core.app.NotificationCompat
+import androidx.core.content.ContextCompat
+import com.vitorpamplona.amethyst.R
+import com.vitorpamplona.amethyst.ui.MainActivity
+
+/**
+ * Process-anchor for an active audio-room session. Holds a partial wake-lock
+ * and a foreground notification so playback continues when the screen is off
+ * or the user briefly leaves the app.
+ *
+ * Scope decisions:
+ *   - The service does NOT own the MoQ session / decoder / player. Those
+ *     live in `AudioRoomViewModel`. This service exists to keep the process
+ *     alive while audio is in flight; the VM still drives all wire activity.
+ *   - Foreground type is `mediaPlayback`; if the user starts broadcasting,
+ *     the screen calls [promoteToMicrophone] which re-`startForeground`s
+ *     with `mediaPlayback|microphone` (Android 14+ requires the explicit
+ *     microphone type while the mic is open).
+ *   - One service per process — multiple audio-room screens are not a real
+ *     scenario today (there's one room per screen and the room screen is a
+ *     full-screen activity). The service uses a shared notification id so
+ *     start/promote/stop is idempotent.
+ */
+class AudioRoomForegroundService : Service() {
+    private var wakeLock: PowerManager.WakeLock? = null
+    private var promoted = false
+
+    override fun onCreate() {
+        super.onCreate()
+        createNotificationChannel()
+        wakeLock =
+            (getSystemService(Context.POWER_SERVICE) as PowerManager)
+                .newWakeLock(PowerManager.PARTIAL_WAKE_LOCK, "amethyst:audio-room")
+                .apply { setReferenceCounted(false) }
+    }
+
+    override fun onStartCommand(
+        intent: Intent?,
+        flags: Int,
+        startId: Int,
+    ): Int {
+        when (intent?.action) {
+            ACTION_PROMOTE_TO_MIC -> {
+                startForegroundWithType(includeMic = true)
+            }
+
+            ACTION_STOP -> {
+                stopForeground(STOP_FOREGROUND_REMOVE)
+                stopSelf()
+            }
+
+            else -> {
+                startForegroundWithType(includeMic = promoted)
+            }
+        }
+        if (wakeLock?.isHeld != true) {
+            wakeLock?.acquire(WAKE_LOCK_TIMEOUT_MS)
+        }
+        return START_STICKY
+    }
+
+    private fun startForegroundWithType(includeMic: Boolean) {
+        promoted = includeMic
+        val notification = buildNotification()
+        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.UPSIDE_DOWN_CAKE) {
+            val type =
+                if (includeMic) {
+                    ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PLAYBACK or
+                        ServiceInfo.FOREGROUND_SERVICE_TYPE_MICROPHONE
+                } else {
+                    ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PLAYBACK
+                }
+            startForeground(NOTIFICATION_ID, notification, type)
+        } else {
+            startForeground(NOTIFICATION_ID, notification)
+        }
+    }
+
+    private fun buildNotification(): Notification {
+        val openIntent =
+            PendingIntent.getActivity(
+                this,
+                0,
+                Intent(this, MainActivity::class.java),
+                PendingIntent.FLAG_IMMUTABLE or PendingIntent.FLAG_UPDATE_CURRENT,
+            )
+        val stopIntent =
+            PendingIntent.getService(
+                this,
+                1,
+                Intent(this, AudioRoomForegroundService::class.java).apply { action = ACTION_STOP },
+                PendingIntent.FLAG_IMMUTABLE or PendingIntent.FLAG_UPDATE_CURRENT,
+            )
+
+        val title =
+            if (promoted) {
+                getString(R.string.audio_room_notification_broadcasting)
+            } else {
+                getString(R.string.audio_room_notification_listening)
+            }
+
+        return NotificationCompat
+            .Builder(this, CHANNEL_ID)
+            .setContentTitle(title)
+            .setContentText(getString(R.string.audio_room_notification_text))
+            .setSmallIcon(R.drawable.amethyst)
+            .setOngoing(true)
+            .setContentIntent(openIntent)
+            .addAction(0, getString(R.string.audio_room_notification_stop), stopIntent)
+            .setCategory(NotificationCompat.CATEGORY_CALL)
+            .build()
+    }
+
+    private fun createNotificationChannel() {
+        val mgr = getSystemService(Context.NOTIFICATION_SERVICE) as NotificationManager
+        if (mgr.getNotificationChannel(CHANNEL_ID) == null) {
+            mgr.createNotificationChannel(
+                NotificationChannel(
+                    CHANNEL_ID,
+                    getString(R.string.audio_room_notification_channel),
+                    NotificationManager.IMPORTANCE_LOW,
+                ).apply {
+                    description = getString(R.string.audio_room_notification_channel_description)
+                    setShowBadge(false)
+                },
+            )
+        }
+    }
+
+    override fun onDestroy() {
+        wakeLock?.takeIf { it.isHeld }?.release()
+        wakeLock = null
+        super.onDestroy()
+    }
+
+    override fun onBind(intent: Intent?): IBinder? = null
+
+    companion object {
+        private const val CHANNEL_ID = "audio_room_foreground"
+        private const val NOTIFICATION_ID = 0xA0D10
+        private const val ACTION_PROMOTE_TO_MIC = "com.vitorpamplona.amethyst.audio_room.PROMOTE_MIC"
+        private const val ACTION_STOP = "com.vitorpamplona.amethyst.audio_room.STOP"
+        private const val WAKE_LOCK_TIMEOUT_MS = 12L * 60 * 60 * 1000 // 12 hours hard cap
+
+        /** Start the service in listener-only foreground mode. Idempotent. */
+        fun startListening(context: Context) {
+            ContextCompat.startForegroundService(
+                context,
+                Intent(context, AudioRoomForegroundService::class.java),
+            )
+        }
+
+        /**
+         * Re-`startForeground` with mediaPlayback + microphone type while the
+         * user is broadcasting. Required by Android 14's split foreground-type
+         * permission model.
+         */
+        fun promoteToMicrophone(context: Context) {
+            ContextCompat.startForegroundService(
+                context,
+                Intent(context, AudioRoomForegroundService::class.java).apply {
+                    action = ACTION_PROMOTE_TO_MIC
+                },
+            )
+        }
+
+        /** Stop and remove the foreground notification. Idempotent. */
+        fun stop(context: Context) {
+            context.stopService(Intent(context, AudioRoomForegroundService::class.java))
+        }
+    }
+}
@@ -68,6 +68,7 @@ import com.vitorpamplona.amethyst.commons.model.nip53LiveActivities.LiveActiviti
 import com.vitorpamplona.amethyst.commons.viewmodels.AudioRoomViewModel
 import com.vitorpamplona.amethyst.commons.viewmodels.BroadcastUiState
 import com.vitorpamplona.amethyst.commons.viewmodels.ConnectionUiState
+import com.vitorpamplona.amethyst.service.audiorooms.AudioRoomForegroundService
 import com.vitorpamplona.amethyst.ui.note.ClickableUserPicture
 import com.vitorpamplona.amethyst.ui.note.LoadAddressableNote
 import com.vitorpamplona.amethyst.ui.screen.loggedIn.AccountViewModel
@@ -95,20 +96,19 @@ import kotlinx.coroutines.launch
 * Clubhouse-style audio-room "stage" rendered in place of the video player when
 * the underlying activity is a NIP-53 kind 30312 [MeetingSpaceEvent].
 *
- * Responsibilities (M1 = listener-only):
+ * Responsibilities:
 *   - Displays host / speaker / audience avatars parsed from the 30312 `p` tags.
- *   - Publishes kind 10312 presence on enter and every 30 s while composed.
- *   - Hand-raise toggle flips the `["hand","1"|"0"]` tag on that presence event
- *     so a host on any NIP-53 client can see the request and promote.
- *   - **Connect button** opens the listener-side audio pipeline (HTTP →
- *     WebTransport over QUIC → MoQ → Opus decode → AudioTrack). On Connected,
- *     auto-subscribes to every host's speaker track.
- *   - **Mute toggle** silences the local audio device without halting the
- *     network pipeline so unmute is instant.
- *
- * Speaker-side (mic capture + publish) is M5+ in the audio-rooms completion
- * plan — `nestsClient/plans/2026-04-26-audio-rooms-completion.md`. Until then
- * the presence event omits the `muted` tag (we have no mic to mute).
+ *     Avatars get a primary-color ring while their MoQ track is delivering audio.
+ *   - Publishes kind 10312 presence on enter and every 30 s while composed,
+ *     reflecting the hand-raise + mic-mute state.
+ *   - **Listener** path: Connect button → HTTP → WebTransport over QUIC → MoQ
+ *     listener session → Opus decode → AudioTrack. Auto-subscribes to every
+ *     host + speaker track. Mute toggle silences the local device without
+ *     halting the network so unmute is instant.
+ *   - **Speaker** path (only for users in the room's `p` tags as host or
+ *     speaker): Talk button → RECORD_AUDIO permission → second MoQ session
+ *     in publisher mode → AudioRecord → MediaCodec Opus encoder → MoQ
+ *     OBJECT_DATAGRAM emission. Live indicator + mic-mute toggle.
 */
@Composable
 fun AudioRoomStage(
@@ -144,25 +144,6 @@ private fun AudioRoomStageContent(
    val scope = rememberCoroutineScope()
    val account = accountViewModel.account

-    // Publish initial presence on enter and refresh every PRESENCE_REFRESH_MS while composed.
-    LaunchedEffect(event.address().toValue(), handRaised) {
-        publishPresence(account, event, handRaised)
-        while (isActive) {
-            delay(PRESENCE_REFRESH_MS)
-            publishPresence(account, event, handRaised)
-        }
-    }
-
-    // Best-effort "leave" — re-publish a lowered-hand presence so peers see us
-    // drop sooner than the 30 s heartbeat would otherwise allow.
-    DisposableEffect(event.address().toValue()) {
-        onDispose {
-            scope.launch(Dispatchers.IO) {
-                runCatching { publishPresence(account, event, handRaised = false) }
-            }
-        }
-    }
-
    val serviceBase = event.service()
    val roomId = event.address().dTag
    val audioAvailable = !serviceBase.isNullOrBlank() && roomId.isNotBlank()
@@ -193,6 +174,52 @@ private fun AudioRoomStageContent(
    val ui = viewModel?.uiState?.collectAsState()?.value
    val speakingNow = ui?.speakingNow ?: persistentSetOf()

+    // Foreground-service lifecycle. The service is a process-anchor — it
+    // doesn't own the audio resources (those live in the VM) but its
+    // foreground notification + wake-lock keep audio playing with the
+    // screen off. Promotes to mediaPlayback+microphone type while the user
+    // broadcasts (Android 14+ split foreground-type permission model).
+    val context = LocalContext.current
+    val isConnected = ui?.connection is ConnectionUiState.Connected
+    val isBroadcasting = ui?.broadcast is BroadcastUiState.Broadcasting
+    LaunchedEffect(isConnected, isBroadcasting) {
+        when {
+            isConnected && isBroadcasting -> AudioRoomForegroundService.promoteToMicrophone(context)
+            isConnected -> AudioRoomForegroundService.startListening(context)
+            else -> AudioRoomForegroundService.stop(context)
+        }
+    }
+    DisposableEffect(Unit) {
+        onDispose { AudioRoomForegroundService.stop(context) }
+    }
+    // Mic state for the presence event: null when not broadcasting (we have
+    // no mic stream to be muted on), explicit true/false while live so other
+    // clients can render the mute indicator on our avatar.
+    val micMutedTag: Boolean? =
+        when (val b = ui?.broadcast) {
+            is BroadcastUiState.Broadcasting -> b.isMuted
+            else -> null
+        }
+
+    // Publish initial presence on enter and refresh every PRESENCE_REFRESH_MS while composed.
+    LaunchedEffect(event.address().toValue(), handRaised, micMutedTag) {
+        publishPresence(account, event, handRaised, micMutedTag)
+        while (isActive) {
+            delay(PRESENCE_REFRESH_MS)
+            publishPresence(account, event, handRaised, micMutedTag)
+        }
+    }
+
+    // Best-effort "leave" — re-publish a lowered-hand presence so peers see us
+    // drop sooner than the 30 s heartbeat would otherwise allow.
+    DisposableEffect(event.address().toValue()) {
+        onDispose {
+            scope.launch(Dispatchers.IO) {
+                runCatching { publishPresence(account, event, handRaised = false, micMuted = null) }
+            }
+        }
+    }
+
    Card(
        modifier = Modifier.fillMaxWidth().padding(8.dp),
        shape = RoundedCornerShape(12.dp),
@@ -530,16 +557,17 @@ private suspend fun publishPresence(
    account: com.vitorpamplona.amethyst.model.Account,
    event: MeetingSpaceEvent,
    handRaised: Boolean,
+    micMuted: Boolean?,
 ) {
    runCatching {
        account.signAndComputeBroadcast(
            MeetingRoomPresenceEvent.build(
                root = event,
                handRaised = handRaised,
-                // muted tag intentionally omitted — listener-only mute (M1)
-                // silences our speakers, not a mic we don't yet broadcast.
-                // Re-evaluate when M5 (publisher path) lands.
-                muted = null,
+                // null = not broadcasting (no mic stream); true/false reflect
+                // the speaker's current mic state so other clients can show a
+                // mute indicator on the avatar.
+                muted = micMuted,
            ),
        )
    }
@@ -512,6 +512,12 @@
    <string name="audio_room_broadcasting">Live</string>
    <string name="audio_room_broadcast_failed">Broadcast failed: %1$s</string>
    <string name="audio_room_record_permission_required">Microphone access is required to talk in this room.</string>
+    <string name="audio_room_notification_channel">Audio rooms</string>
+    <string name="audio_room_notification_channel_description">Keeps audio playing while a room is open.</string>
+    <string name="audio_room_notification_listening">Audio room connected</string>
+    <string name="audio_room_notification_broadcasting">Audio room — Live</string>
+    <string name="audio_room_notification_text">Tap to return.</string>
+    <string name="audio_room_notification_stop">Stop</string>
    <string name="longs">Videos</string>
    <string name="articles">Articles</string>
    <string name="private_bookmarks">Private Bookmarks</string>
@@ -109,6 +109,12 @@ class AudioRoomViewModel(
    private var requestedSpeakers: Set<String> = emptySet()
    private var closed = false

+    // Auto-reconnect state — incremented every time we observe Failed and
+    // schedule a retry; reset to 0 on Connected or on a user-initiated
+    // connect()/disconnect() so manual recovery starts fresh.
+    private var autoRetryAttempts = 0
+    private var autoRetryJob: Job? = null
+
    // Speaker / publisher path
    private var speaker: NestsSpeaker? = null
    private var broadcastHandle: BroadcastHandle? = null
@@ -133,6 +139,11 @@ class AudioRoomViewModel(
        val current = _uiState.value.connection
        if (current is ConnectionUiState.Connecting || current is ConnectionUiState.Connected) return

+        // User-initiated connect — clear any pending auto-retry timer + counter.
+        autoRetryJob?.cancel()
+        autoRetryJob = null
+        autoRetryAttempts = 0
+
        _uiState.update { it.copy(connection = ConnectionUiState.Connecting(ConnectionUiState.Step.ResolvingRoom)) }

        connectJob =
@@ -264,12 +275,17 @@ class AudioRoomViewModel(
    /** Tear down without finalizing the VM (e.g. user pressed Disconnect). */
    fun disconnect() {
        if (closed) return
+        autoRetryJob?.cancel()
+        autoRetryJob = null
+        autoRetryAttempts = 0
        teardownBroadcast(BroadcastUiState.Idle)
        teardown(targetState = ConnectionUiState.Idle)
    }

    override fun onCleared() {
        closed = true
+        autoRetryJob?.cancel()
+        autoRetryJob = null
        teardownBroadcast(BroadcastUiState.Idle)
        teardown(targetState = ConnectionUiState.Closed)
        super.onCleared()
@@ -323,9 +339,83 @@ class AudioRoomViewModel(
                    _uiState.update { ui ->
                        ui.copy(connection = state.toUiState(ui.connection))
                    }
-                    if (state is NestsListenerState.Connected) {
+                    when (state) {
+                        is NestsListenerState.Connected -> {
+                            autoRetryAttempts = 0
                            reconcileSubscriptions()
                        }
+
+                        is NestsListenerState.Failed -> {
+                            scheduleAutoRetry()
+                        }
+
+                        else -> { /* no extra side effect */ }
+                    }
+                }
+            }
+    }
+
+    /**
+     * Auto-reconnect on listener Failed with capped exponential backoff.
+     * The user can still tap Connect manually at any time; that resets the
+     * retry counter via the `connect()` path.
+     */
+    private fun scheduleAutoRetry() {
+        if (closed) return
+        if (autoRetryJob?.isActive == true) return
+        if (autoRetryAttempts >= MAX_AUTO_RETRIES) return
+
+        val attempt = autoRetryAttempts
+        autoRetryAttempts = attempt + 1
+        val backoffMs = minOf(MAX_RETRY_BACKOFF_MS, INITIAL_RETRY_BACKOFF_MS shl attempt)
+        autoRetryJob =
+            viewModelScope.launch {
+                delay(backoffMs)
+                if (closed) return@launch
+                if (_uiState.value.connection !is ConnectionUiState.Failed) return@launch
+                // Drop the previous listener cleanly before starting a new
+                // attempt. Reset state to Idle so the connect() guard passes.
+                teardown(targetState = ConnectionUiState.Idle)
+                connectInternal()
+            }
+    }
+
+    /**
+     * Internal connect that bypasses the auto-retry counter reset — used by
+     * [scheduleAutoRetry] so successive auto-retries continue to back off.
+     */
+    private fun connectInternal() {
+        if (closed) return
+        val current = _uiState.value.connection
+        if (current is ConnectionUiState.Connecting || current is ConnectionUiState.Connected) return
+
+        _uiState.update { it.copy(connection = ConnectionUiState.Connecting(ConnectionUiState.Step.ResolvingRoom)) }
+
+        connectJob =
+            viewModelScope.launch {
+                try {
+                    val l =
+                        connector.connect(
+                            httpClient = httpClient,
+                            transport = transport,
+                            scope = viewModelScope,
+                            serviceBase = serviceBase,
+                            roomId = roomId,
+                            signer = signer,
+                        )
+                    if (closed) {
+                        runCatching { l.close() }
+                        return@launch
+                    }
+                    listener = l
+                    observeListenerState(l)
+                } catch (ce: CancellationException) {
+                    throw ce
+                } catch (t: Throwable) {
+                    _uiState.update {
+                        it.copy(connection = ConnectionUiState.Failed(t.message ?: t::class.simpleName ?: "connect failed"))
+                    }
+                    scheduleAutoRetry()
                }
            }
    }
@@ -588,6 +678,15 @@ sealed class BroadcastUiState {
 */
 const val SPEAKING_TIMEOUT_MS: Long = 250L

+/** Max number of auto-reconnect attempts after a Failed listener state. */
+private const val MAX_AUTO_RETRIES = 3
+
+/** First auto-retry backoff in ms; doubles each subsequent attempt. */
+private const val INITIAL_RETRY_BACKOFF_MS = 1_000L
+
+/** Cap on the auto-retry backoff so the wait stays human-acceptable. */
+private const val MAX_RETRY_BACKOFF_MS = 16_000L
+
 /**
 * Indirection over the top-level `connectNestsListener` so tests can drive
 * a fake [NestsListener] directly without standing up an HTTP fake +