be4e0b9f98177906a20c212c8867d223e13b525b
13 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Claude
|
be4e0b9f98 |
fix(nests): T12 carry audio group sequence across hot-swaps
Every JWT-refresh hot-swap reset the audio publisher's group sequence
counter to 0, because `PublisherStateImpl` always initialised
`nextSequence = 0L`. kixelated/hang's `Container.Consumer.#run`
discards any consumer with `sequence < this.#active`, so a watcher
whose `#active` had advanced to e.g. 50 from the previous session's
publisher would silently drop every group on the new session until
either `#active` rolls over or the watcher re-subscribes — audible as
"speaker goes silent for a minute every 9 minutes" (the proactive
JWT refresh window).
Carry the sequence forward end-to-end:
- MoqLiteSession.publish gains a `startSequence: Long = 0L`
parameter; PublisherStateImpl seeds `nextSequenceField` from it
instead of hard-coding 0.
- MoqLitePublisherHandle exposes a `nextSequence: Long` snapshot.
`@Volatile`-backed so the hot-swap caller can read it without
contending the publisher's gate; race-window between read and
a concurrent send is sub-millisecond and would only produce a
single duplicate sequence in the very unlikely overlap, which
listeners tolerate.
- HotSwappablePublisherSource.openPublisherForHotSwap takes
`startSequence: Long = 0L`. MoqLiteNestsSpeaker passes it
through to session.publish.
- NestMoqLiteBroadcaster exposes its current publisher via a
read-only `currentPublisher` so the hot-swap pump in
ReconnectingNestsSpeaker.runHotSwapIteration can read its
`nextSequence` before opening the replacement publisher and
pass it as the seed.
- Catalog publisher keeps `startSequence = 0L` (the default) —
catalog isn't subject to the same `#active` accumulation
because each new SUBSCRIBE triggers a fresh emit-on-subscribe
write that resets the watcher's catalog state.
Listener-side change is none — the watcher already reads whatever
sequence we send. A new MoqLiteSessionTest pins the contract:
publishing with startSequence=42 makes the first uni stream's
GroupHeader.sequence == 42 and advances to 43 after the first send.
https://claude.ai/code/session_014JfZJHSTvyYYWJbC9VbB47
|
||
|
Claude
|
3417e44a6a |
refactor(nests): catalog emit-on-subscribe hook replaces 2s republish loop
The catalog publisher's previous shape was "send once at startup
(silently fails if no subs are attached yet) + a 2 s republish loop".
That worked but had two problems:
1. The startup send is a no-op (PublisherStateImpl.send returns false
when inboundSubs is empty), so the catalog isn't actually on the
wire until the first loop tick — up to 2 s of catalog dead time
for a watcher that subscribes immediately.
2. Re-emitting a static blob every 2 s for the broadcast's lifetime
wastes a fresh group on the relay's per-track cache; not a hot path
today but unnecessary.
Replace with an emit-on-subscribe hook. New API on MoqLitePublisherHandle:
fun setOnNewSubscriber(hook: (suspend () -> Unit)?)
PublisherStateImpl fires the hook once per accepted (track-matching)
inbound SUBSCRIBE, OUTSIDE its serialisation lock so the hook can
safely call send/endGroup without deadlock. Track-mismatched subs
(SubscribeDrop reply) do NOT fire the hook — covered by a new
"hook does not fire on track mismatch" test.
MoqLiteNestsSpeaker.startBroadcasting and ReconnectingNestsSpeaker's
hot-swap iteration both now register a hook that writes the catalog
JSON on every fresh subscribe instead of looping. The
catalogRepublishJob field on MoqLiteBroadcastHandle and the
hotSwapCatalogJob field on ReissuingBroadcastHandle (plus their close
paths) are dropped.
Net effect for hang.js / NestsUI-v2 watchers:
- Catalog reaches the watcher on the SAME group as the subscribe ack
(no 0–2 s startup gap).
- Late joiners hit the relay's track-latest cache for the catalog
blob — a fresh hook fire only happens when the relay opens a new
SUBSCRIBE bidi to us (cliff-detector recycle on listener side, or
JWT-refresh on our side).
- One catalog group per relay-side subscribe instead of one every
2 s for the broadcast lifetime.
Two tests pin the new behaviour: hook fires on inbound subscribe;
hook does NOT fire on a track-mismatched subscribe (which gets a
SubscribeDrop reply and never enters inboundSubs).
https://claude.ai/code/session_014JfZJHSTvyYYWJbC9VbB47
|
||
|
Claude
|
ff9bb25921 |
refactor(nests): replace hand-built catalog JSON with typed MoqLiteHangCatalog
The two SPEAKER_CATALOG_JSON call sites (MoqLiteNestsSpeaker for the non-reconnecting path, ReconnectingNestsSpeaker for the JWT-refresh hot- swap path) both built the manifest as a `\\` + `\"` + `+`-concatenated string literal. One missed escape and the watcher's parser fails silently — the broadcast becomes invisible to hang.js with no indication of why. The two literals had also drifted independently in review (same content today; nothing prevents drift tomorrow). Replace both with `MoqLiteHangCatalog.opusMono48k(track).encodeJsonBytes()`, a Serializable data class that encodes via kotlinx.serialization with `encodeDefaults = false` + `explicitNulls = false` pinned (so a future global default-flip can't surface `"bitrate": null` on hang.js's parser). The new type lives in :nestsClient because :nestsClient does not depend on :commons and the parser-side `RoomSpeakerCatalog` (in :commons) isn't reachable from the publish path. The two classes target the same wire shape independently; a byte-exact assertion in the new MoqLiteHangCatalogTest plus the existing RoomSpeakerCatalogTest's inline-literal round-trip together pin both sides — desync on either end fails one of the two tests immediately. https://claude.ai/code/session_014JfZJHSTvyYYWJbC9VbB47 |
||
|
Claude
|
5b7e347ac0 |
feat(nests): align moq-lite catalog with kixelated/hang shape
Switch RoomSpeakerCatalog to the canonical hang catalog format (camelCase audio.renditions[<track>] with container.kind), wrap audio frames in the legacy varint(timestamp_us) prefix on publish, and strip it on subscribe. Also reply to inbound Probe/Fetch/Session bidis with a bitrate hint or clean FIN instead of hanging. |
||
|
Claude
|
babf7623d8 |
feat(nests): publish catalog.json so browsers can discover broadcasts
Two-phone Amethyst sessions stream Opus frames just fine on the wire,
but the kixelated/moq browser watcher (the canonical web reference) sees
nothing — because we never published a catalog.json track. moq-lite
watchers discover a broadcast's tracks by subscribing to the
`catalog.json` track on the broadcast suffix and parsing the JSON
manifest. Without it the watcher has nothing to subscribe to.
Two parts:
1. MoqLiteSession.publish now supports multiple tracks per session,
sharing one broadcast suffix. The previous "one publisher per
session" invariant rejected the second publish() with
"already publishing", forcing a single track per session.
2. MoqLiteNestsSpeaker.startBroadcasting now opens both an audio
publisher AND a catalog publisher, pushes the manifest once, and
launches a 2s republish loop so late-attaching watchers don't wait
indefinitely for the next refresh.
The manifest shape mirrors RoomSpeakerCatalog (kotlinx.serialization
data class on the listener side):
{"version":1,"audio":[{"track":"audio/data","codec":"opus",
"sample_rate":48000,"channel_count":1}]}
Hard-coded for now since OpusEncoder is fixed at 48kHz mono.
Hot-swap path (ReconnectingNestsSpeaker session refresh) is NOT yet
wired to recycle the catalog publisher — acceptable trade-off for a
diagnostic landing; follow-up will move the catalog republisher into
HotSwappablePublisherSource so it survives JWT refreshes.
|
||
|
Claude
|
076b301d84 |
fix(nests): close 4 audio-dropout sources across listener + speaker
1. Listener-side pre-roll + bigger playback buffer + audio-priority thread.
- NestPlayer buffers 5 decoded frames (~100 ms) before starting
the AudioPlayer, masking the first-frame underrun that fires
whenever Compose / GC briefly stalls Main.
- AudioTrackPlayer sizes the AudioTrack at max(minBuffer*16, 250 ms)
instead of minBuffer*4 (~80 ms) and writes via a per-instance
audio-priority single-thread executor (Process.THREAD_PRIORITY_AUDIO)
instead of Dispatchers.IO, so WRITE_BLOCKING never contends with
unrelated IO work.
2. MoqLiteSession.subscribe: hoist response typeCode out of the
collect lambda. readVarint advances pos permanently while
readSizePrefixed only rolls back its own length-varint, so a
chunk-split between type and body would re-read the body's size
prefix as the type code on the next chunk and misframe the
response. Mirrors the same fix already in handleInboundBidi.
3. MoqLiteSession.subscribe: register the subscription in the map
BEFORE writing the SUBSCRIBE bytes on the wire. The relay can
open the first group's uni stream before our continuation
re-enters [state] to register; if so, drainOneGroup looked the
id up against an empty map and silently dropped the frame
(~1 frame / 20 ms gap on first attach). Wrap the post-register
writes so a transport-failure unwinds the orphan registration.
4. Hot-swap moq-lite publisher across JWT-refresh boundaries.
- NestMoqLiteBroadcaster: publisher is now @Volatile + supports
swapPublisher(); capture loop snapshots the reference per frame
and resets framesInCurrentGroup on swap.
- MoqLiteNestsSpeaker implements a new internal
HotSwappablePublisherSource interface that exposes
openPublisherForHotSwap(track) without spinning up a broadcaster.
- ReissuingBroadcastHandle keeps a single long-lived broadcaster
across session recycles when the speaker supports hot swap;
legacy IETF / fake speakers fall back to close-then-restart.
- connectReconnectingNestsSpeaker.orchestrator hoists the old-
session close onto a sibling launch so the wrapper can swap
the publisher into the broadcaster on the new session before
the old session's WebTransport drops. Eliminates the previously-
accepted 50–150 ms audible silence at every JWT refresh.
|
||
|
Claude
|
43720dd14d |
fix(nests): scope moq-lite publisher to a single track to stop catalog hijack
Listeners stuck on a spinning speaker avatar with no audio (against
both Amethyst-hosted and nostrnests.com-hosted Nests). The trace
captured on a working speaker phone showed every group keyed to
subId=1 / track='catalog.json' even though the broadcaster was
pumping Opus frames:
13:21:27.570 inbound SUBSCRIBE id=1 track='catalog.json'
13:21:27.574 inbound SUBSCRIBE id=0 track='audio/data'
13:21:27.609 openGroup seq=0 keyedOnSubId=1 track='catalog.json'
13:21:27.610 broadcaster: send accepted (subscriber attached)
... every subsequent group keyed=1, every audio frame on the
catalog stream
Root cause: `PublisherStateImpl.openNextGroupLocked` keyed each uni
stream off `inboundSubs.first()` with no track filter. The kdoc
asserted "Inbound subscription set is expected to be small (1 in
nests's listener-per-room model)" — that was true when listeners
only opened the audio sub. Once the listener side started opening a
catalog sub alongside (commits ~Apr 2026), whichever SUBSCRIBE
arrived first (typically the catalog one, by ~4 ms in this trace)
became the routing target for all audio frames. The relay forwarded
those frames to the listener with subscribeId=catalog, the listener
routed them to its catalog handler, RoomSpeakerCatalog.parseOrNull
returned null (it's not JSON), and the audio subscription's frames
channel never received anything — so onSpeakerActivity never fired
and the spinner stayed forever.
Fix: per moq-lite Lite-03, a publisher is responsible for one
(broadcast, track) tuple. Thread `track` through `MoqLiteSession.publish`
and have `PublisherStateImpl.registerInboundSubscription` reject
inbound SUBSCRIBEs whose track doesn't match — those still receive
SUBSCRIBE_OK from the bidi handler (best-effort per Lite-03's
optional-content semantics for catalog) but don't influence the
audio routing. `MoqLiteNestsSpeaker.startBroadcasting` passes
`track = MoqLiteNestsListener.AUDIO_TRACK` (= "audio/data").
Test callsites (MoqLiteSessionTest, NostrnestsProdAudioTransmissionTest,
NostrNestsSustainedSendOutcomesInteropTest) updated to pass the new
required parameter; all use track="audio/data" matching what they
exercise.
The diagnostic logs from the prior commits stay in — they're how we
caught this and they'll catch the next one. They can be stripped in
a follow-up once the fix is confirmed in the field.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
|
||
|
Claude
|
5d955eff99 |
feat: animate local speaker ring from MoQ frame transmission
Wire the local user's avatar into the existing speaking-ring animation so the host/speaker sees the same green ring + amplitude glow on their own cell that remote speakers already get. Ground truth is the broadcaster's `publisher.send(opus)` success: the new `onLevel` callback fires only after a frame actually leaves on the wire, computed from the raw PCM peak (shared `peakAmplitude` util used by both the player decode loop and the broadcaster capture loop). This means the animation reflects what the relay sees, not any UI-side mute / role state that could be stale or buggy — if frames are flowing, the ring plays. Plumbing: `NestsSpeaker.startBroadcasting` gains an optional `onLevel` that the moq-lite + IETF broadcasters both honour, the reconnecting wrapper replays it on every session reissue (alongside the existing mute-intent replay), and `NestViewModel.startBroadcast` hands it a lambda that calls the existing `onSpeakerActivity` / `onAudioLevel` plumbing keyed on the local pubkey. The 250 ms speaking-timeout fades the ring naturally when the user mutes or stops broadcasting. |
||
|
Claude
|
8b7785a7e8 |
fix(nestsClient): token-parse hardening, mint timeout, IPv6 [], broadcaster-bail signal
- NestsTokenResponse.parse catch list now includes IllegalStateException so a malformed escape from a misbehaving auth server surfaces as NestsException instead of crashing the listener. - OkHttpNestsClient gains a configurable callTimeoutMs (default 90 s) enforcing an upper bound on the entire mintToken round-trip including retries. Without it a stalled server suspends the reconnect orchestrator indefinitely (the orchestrator's openOnce step parks on mintToken). 90 s leaves headroom over the worst-case 63 s 429 retry chain documented in MAX_RATE_LIMIT_RETRIES. - parseEndpoint tightens IPv6 bracket check from `closeBracket > 0` to `> 1`, rejecting the empty `[]` literal. - NestBroadcaster + NestMoqLiteBroadcaster gain an `onTerminalFailure` callback that fires once when the consecutive-send-error guard bails. MoqLiteNestsSpeaker wires this to flip the speaker state to Failed, giving ReconnectingNestsSpeaker the signal it needs to recycle the session — without this hook the broadcaster bailed silently and the outward speaker state stayed on Broadcasting forever. Adds regression test: - onTerminalFailure_fires_once_after_consecutive_send_failures 225 tests pass, 0 failures. Android target compiles clean. |
||
|
Claude
|
702885f4cb |
fix(nestsClient): fix race + leaks + Android encoder/decoder bugs from second audit
A fresh audit caught a real race in the round-1 subscribe() rewrite plus several distinct bugs in the Android MediaCodec layer. **MoqLiteSession.subscribe() race regression** — the launched collector that reads the SubscribeResponse and watches for peer FIN ran cleanup (remove from subscriptionsBySubscribeId, close frames) before subscribe() itself reached the post-await registration. If the publisher FIN'd immediately after sending Ok, the collector exited first against an empty map; subscribe() then registered the subscription, leaving frames in the map with no live collector to ever close it on transport drop. Consumer hung forever — exactly the failure mode round-1 fix #3 was supposed to prevent. Fix: pre-register the subscription BEFORE launching the collector. Drop unused `ok` field from ListenerSubscription. **MoqLiteNestsSpeaker.startBroadcasting publisher leak** — if session.publish() succeeded but broadcaster.start() then threw (mic permission denied, AudioRecord allocation failure), the publisher was never closed and stayed registered as session.activePublisher, permanently blocking subsequent startBroadcasting calls. **runCatching{suspend close} swallowing CancellationException** — MoqLiteNestsSpeaker.close, MoqLiteBroadcastHandle.close, MoqLiteNestsListener.close all wrapped suspending closes in runCatching, breaking structured cancellation when the parent scope cancelled teardown. Replaced with explicit cancel-rethrowing try/catch. **MediaCodecOpusDecoder ArrayList<Short> boxing on every audio frame** — at 50 fps × 960 samples × N speakers ≈ 48 000 boxed Short allocations/sec/speaker on the audio hot path. Rewritten to write directly into a pre-sized ShortArray via ShortBuffer.get(dst, off, len). **MediaCodecOpusEncoder bugs** - KEY_AAC_PROFILE = AACObjectLC was being set on an audio/opus encoder. Meaningless for Opus; stricter Codec2 stacks on Android 13+ reject the configure() call with IllegalArgumentException and surface as DeviceUnavailable. Removed. - The drain loop's INFO_OUTPUT_FORMAT_CHANGED branch had no progress guard. A buggy encoder re-emitting FORMAT_CHANGED without producing output would busy-spin against the 10 ms dequeue timeout. Now absorbed at most once per encode call. Same guard added to the decoder. Adds regression test: - frames_flow_completes_when_peer_FINs_immediately_after_Ok 224 tests pass, 0 failures. Android target compiles clean. |
||
|
Claude
|
10ad69f1af |
nestsClient: pack 5 Opus frames per moq-lite group to dodge the prod stream cliff
Production sweep against nostrnests.com cliffed at exactly 99 frames received
across every multi-frame scenario (frames200/400, 30s, 120s, etc.) when the
broadcaster opened one QUIC uni stream per Opus frame. The investigation in
nestsClient/plans/2026-05-01-quic-stream-cliff-investigation.md walks through
why the obvious "raise the stream cap" hypothesis didn't pan out (publisher.send
was never returning false past the cap, the local QuicConnection was already
seeing the relay extend MAX_STREAMS_UNI dynamically).
The same sweep showed sweep_frames_per_group_5 / _20 / _all delivering 100/100
frames cleanly. The cliff scales with stream-creation rate, not byte volume or
total stream count — so packing N frames per group (one uni stream per N frames
instead of per frame) bypasses whatever the underlying limit is.
NestMoqLiteBroadcaster:
- Add framesPerGroup constructor param, default 5 (≈ 100 ms of audio per
moq-lite group). Stream-creation rate drops from 50/sec to 10/sec, well
below the production cliff threshold.
- Counter resets after each endGroup; trailing partial group flushes its
FIN on capture EOF so the listener doesn't sit on a half-open stream.
MoqLiteNestsSpeaker + connectNestsSpeaker:
- Plumb framesPerGroup through to the broadcaster. Default flows from
NestMoqLiteBroadcaster.DEFAULT_FRAMES_PER_GROUP so production callers
get the mitigation without code changes; tests can override to 1 to
keep their groupId-per-frame assertions.
Tests:
- All harness interop tests (RoundTrip, LateJoin, MultiPeer, Mute,
Unsubscribe, SpeakerClose, both Reconnecting variants) plus the prod
NostrnestsProdAudioTransmissionTest now pass framesPerGroup = 1
explicitly to preserve their per-frame group/object id assertions.
- All :quic and :nestsClient JVM tests pass.
Trade-off: a brand-new subscriber that joins mid-broadcast picks up at the
next group boundary. With 5 frames/group the late-join initial gap is up
to 100 ms (perceptually inaudible for live audio). The previous wire shape
gave at most 20 ms.
Investigation plan for the underlying QUIC cliff lives in
nestsClient/plans/2026-05-01-quic-stream-cliff-investigation.md. The
attempted-then-reverted suspend/STREAMS_BLOCKED commit is summarised
under "Attempt 1" in that doc.
|
||
|
Claude
|
dc3ac31ae4 |
refactor: rename Audio Room → Nest project-wide
Aligns class names, package paths, string resource keys, UI text and intent actions with the Nests branding used by the EGG specs in nestsClient/specs/. Mechanical rename — no behavior change. - Folders: audiorooms/ → nests/ (5 paths across amethyst, quartz) - 30+ class renames (AudioRoom* → Nest*, AudioRooms* → Nests*) - String resource keys audio_room_* → nest_* - UI strings "Audio Room"/"Audio Rooms" → "Nest"/"Nests" (incl. all locales) - Intent extras AUDIO_ROOM_* → NEST_* - Compose route Route.AudioRooms → Route.Nests Spec-aligned identifiers (MeetingSpaceEvent, meetingSpaces/, the nip53* packages) are intentionally untouched — those are NIP-53 protocol names, not "audio room" branding. https://claude.ai/code/session_01RDpuki4t8StSg1CZcXnV5b |
||
|
Claude
|
71cf99dc22 |
feat(audio-rooms): moq-lite speaker side end-to-end (phase 5c-speaker)
Production speaker path now runs on moq-lite, so connectNestsSpeaker
exchanges real moq-lite framing with the nostrnests reference relay.
Transport layer:
- WebTransportSession.incomingBidiStreams() — peer-initiated bidi
flow. moq-lite publishers receive Announce + Subscribe bidis from
the relay (rs/moq-lite/src/lite/publisher.rs:40 uses
Stream::accept(session)), so the abstraction grew the
accept-bidi-from-peer surface.
- WebTransportSession.openUniStream() — locally-opened uni stream
for group push (rs/moq-lite/src/lite/publisher.rs:338 uses
session.open_uni()).
- :quic WtPeerStreamDemux StrippedWtStream now carries optional
send/finish closures. The demux takes the QuicConnectionDriver
so wakeups fire after each app-level write on a peer-initiated
bidi.
- FakeWebTransport now exposes incomingBidiStreams + openUniStream
directly; the openPeerUniStream test helper went away (production
flow covers it).
Session layer:
- MoqLiteSession.publish(suffix) — claims a broadcast suffix and
lazily launches a relay→us bidi pump. ControlType=Announce reads
AnnouncePlease, replies Active(suffix). ControlType=Subscribe reads
body, replies SubscribeOk, registers the inbound subscription.
- MoqLitePublisherHandle — startGroup / send / endGroup / close
semantics. send opens a uni stream per group with DataType=0 +
GroupHeader and pushes varint(size)+payload frames. close emits
Announce(Ended) on every active announce bidi, FINs the uni.
Application layer:
- AudioRoomMoqLiteBroadcaster — sibling of AudioRoomBroadcaster but
drives MoqLitePublisherHandle (keeps IETF broadcaster intact for
its unit tests).
- MoqLiteNestsSpeaker — NestsSpeaker adapter, mirror of
MoqLiteNestsListener on the publish side.
- connectNestsSpeaker now opens a MoqLiteSession (no SETUP) and
returns MoqLiteNestsSpeaker.
Tests:
- 4 new MoqLiteSessionTest cases:
publisher_replies_to_announcePlease_with_active_announce,
publisher_acks_subscribe_and_pushes_group_data_on_uni_stream,
publisher_send_returns_false_when_no_inbound_subscriber,
publisher_close_emits_ended_announce.
Verified :commons:compileKotlinJvm + :amethyst:compilePlayDebugKotlin
both still compile against the swap.
Docs (plans + CLAUDE.md) refreshed to reflect speaker-side landing.
|