The pre-push hook runs the full `./gradlew test` suite, which doesn't
pass `-DnestsHangInterop=true`. Without that flag the sidecars don't
exist, so every test in this class blew up on `NativeMoqRelayHarness.shared()`
with `IllegalStateException`. Adds a `@BeforeTest` that calls
`assumeHangInterop()` and skips the case via JUnit's Assume — same
pattern `HangInteropTest`, `HangInteropReverseTest`, and the rest of
the `interop/native/` suite already use.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`AudioLatencyComparisonTest`:
- Three new scenarios: 10 %, 20 %, 30 % loss in addition to the
existing clean and 5 %. The whole benchmark now sweeps a useful
arc for finding the breakdown point.
- Frame-count floor now scales with loss instead of a fixed 60 %
threshold. QUIC streams are reliable so loss surfaces mainly as
latency, but at high rates the retransmit + congestion-control
feedback can eat into the 10 s window, so the floor relaxes to
`max(50, (1 - 3*lossRate) * EXPECTED_FRAMES)`. The 50-frame
absolute floor (= 1 s of audio) catches a catastrophic publisher
failure without flapping on the loaded loss-path runs.
Linux benchmark harness (`nestsClient/tests/hang-interop/linux-bench/`):
- `Dockerfile` for an arm64/amd64 image with JDK 21 + rustc 1.95 +
cmake + the *-sys-crate build deps (libssl-dev, build-essential,
pkg-config). Pins rustc 1.95 explicitly so moq-relay v0.10.25's
Cargo.lock-requested `constant_time_eq@0.4.3` builds cleanly.
- `run.sh` wrapper that:
* builds the image,
* unpacks a filtered tarball of the host repo into the container
(mac host `nestsClient/build/` and `cargo target/` stay
untouched),
* symlinks `nestsClient/build` -> `/persistent-gradle-build` and
`nestsClient/tests/hang-interop/target` ->
`/persistent-cargo-target` so successive runs reuse compiled
artefacts (~3-5 min cached vs ~7-15 min cold),
* runs the gradle test invocation and surfaces the JUnit XML
stats to `linux-bench/out/`.
Linux numbers observed (arm64 Docker on a MacBook M2, 10 s window):
| Loss | Stack | p50 | p95 | p99 | max | Frames |
|------|--------|-------|-------|-------|-------|--------|
| 0 % | Kotlin | 61.1 | 63.1 | 65.5 | 79.1 | 573 |
| 0 % | Rust | 100.7 | 102.7 | 104.9 | 140.2 | 495 |
| 5 % | Kotlin | 380.9 | 383.4 | 385.7 | 390.5 | 556 |
| 5 % | Rust | 0.7 | 7.3 | 23.2 | 84.2 | 500 |
| 10 % | Kotlin | 120.5 | 122.4 | 124.0 | 132.6 | 568 |
| 10 % | Rust | 0.5 | 21.9 | 60.4 | 84.4 | 500 |
| 20 % | Kotlin | 159.7 | 164.3 | 170.3 | 183.1 | 566 |
| 20 % | Rust | 1.6 | 61.3 | 101.9 | 144.9 | 499 |
| 30 % | Kotlin | 181.0 | 184.2 | 188.1 | 194.0 | 566 |
| 30 % | Rust | 1.2 | 80.7 | 105.3 | 140.8 | 491 |
Notes for future readers: Rust's p50 collapsing to ~1 ms under loss
is the SUBSCRIBE-late-arrival signal, not "Rust is 100x faster" — by
the time the SUBSCRIBE lands at the publisher, the in-flight
encoder has moved past those early frames, so the listener starts
receiving in near-real-time. Read p95/p99/max for the per-frame
story (Rust expands 7 -> 22 -> 61 -> 80 ms p95 across loss rates).
Frame delivery is stable for both stacks even at 30 % loss on Linux,
unlike the macOS run which hits a 3.4 s Rust catastrophe at 30 %.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Expands `AudioLatencyComparisonTest` from pure inter-arrival jitter
into a real one-way latency measurement against the same
`hang-publish` Rust reference, AND adds a loss-path variant gated
through `udp-loss-shim`.
`hang-publish` (Rust sidecar):
- new `--log-send-times` flag. When set, the publisher prints
`SEND frame=<N> send_t_us=<UNIX_EPOCH_MICROS>` to stdout for
every frame, captured with `SystemTime::now()` immediately
before `frame.encode(group)`. Lock + flush per line because a
piped Rust stdout is fully-buffered by default and the JVM-side
parser needs a live stream. Default off so existing scenarios
(the I7 reconnect test in particular) keep their stderr-only
RUST_LOG output untouched.
Kotlin test:
- `TimestampingOpusEncoder` wraps `JvmOpusEncoder` and stamps an
`Instant.now()` epoch-micros value per `encode(...)` call into a
shared map keyed by a monotonic frame counter. encode runs
immediately before `publisher.send` inside
`NestMoqLiteBroadcaster`, so the captured timestamp is the
closest cross-stack anchor we have to "moment the frame entered
the publisher's outbound buffer" — same anchor the Rust side
uses.
- One-way latency = arrival epoch-micros − send epoch-micros per
matched frame. Both sides read `CLOCK_REALTIME` on linux/macOS,
so values can be compared directly without a sync handshake when
the two processes share a host.
- Frame matching uses `MoqObject.objectId` directly. The listener
layer already synthesises objectId as a per-SUBSCRIPTION
monotonic counter, so it equals the publisher's absolute frame
index; a previous draft of this test did `groupId * 5 + objectId`
and silently aligned against the wrong frames, producing
negative latencies.
- New `under_5pct_packet_loss_pacing_and_one_way_latency` scenario.
Spawns TWO `udp-loss-shim` instances (one per publisher: the
shim latches a single client on first datagram, so a shared shim
would silently swallow the second publisher's traffic), each
forwarding 1:1 to the moq-relay's UDP port modulo a 5 %
bidirectional drop rate. The listener stays on the direct path
so any latency growth comes from publisher-side retransmit /
ack feedback, not listener-side loss.
- Both Kotlin and Rust pinned to `FRAMES_PER_GROUP = 5` (Rust's
default; Kotlin's default is 50). Matched so the per-group
uni-stream open/close cost is the same on both sides — the
comparison is about implementation, not about which stack picked
which group size.
Observed (10 s window, localhost, MacBook Pro M2):
===== Audio publisher comparison: clean (loss=0%) =====
Kotlin speaker ttf=137 ms inter-arrival p50/95/99/max=20.17/20.67/20.88/21.16 ms
one-way p50/95/99/max=80.60/81.31/81.74/83.91 ms
Rust hang-publish ttf=325 ms inter-arrival p50/95/99/max=20.00/21.18/21.49/21.95 ms
one-way p50/95/99/max=100.32/101.45/102.04/168.58 ms
===== Audio publisher comparison: loss-5pct (loss=5%) =====
Kotlin speaker one-way p50/95/99/max=80.50/81.55/82.31/86.43 ms
Rust hang-publish one-way p50/95/99/max=100.44/119.88/122.16/158.98 ms
Reading: Kotlin matches Rust on the clean path within ~20 ms
baseline (and actually sits below it), and absorbs 5 % loss with
only +0.6 ms p99 growth vs Rust's +20 ms. The 80-100 ms baseline
is moq-lite's group-buffer floor with 5-frames/group — protocol,
not stack.
Asserts: each side delivers >= 80 % of expected frames on the clean
path, >= 60 % under loss, and clean-path median inter-arrival sits
in [15, 35] ms. Tail percentiles are printed, not gated.
Gated by `-DnestsHangInterop=true`; both sidecars (hang-publish,
udp-loss-shim) come from `interopBuildSidecars`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds `AudioLatencyComparisonTest` to the hang-interop suite — runs the
Amethyst Kotlin speaker AND the upstream `hang-publish` Rust sidecar
into the same local moq-relay, with one Kotlin listener subscribing to
both. Same room, distinct broadcast suffixes (per-publisher pubkey),
same Opus shape (440 Hz mono, 32 kbps, 20 ms frames), 10 s window.
What it measures (per publisher):
- delivered frame count (sanity floor: 80 % of theoretical max)
- inter-arrival p50 / p95 / p99 / max at the listener — the part
the publisher stack actually controls (mic -> encoder ->
publisher.send -> uni-stream open -> QUIC writer -> wire), since
relay + receive path are identical
- time-to-first-frame (informational; Rust pays process-spawn that
JVM doesn't, so not asserted on)
What it does NOT measure: end-to-end glass-to-ear latency (would need
synchronised send-side timestamps from hang-publish, which means
modifying the Rust sidecar) and loss / congestion (no simulator on
the localhost link). For loss behaviour, see `quic/interop/`.
Asserts: each side delivers >= 80 % of expected frames and each
side's median sits in [15, 35] ms — catches a stalled publisher or
a no-pacing burst-mode publisher without depending on per-host tail
percentiles. Tail numbers are printed for human inspection.
Observed on this run:
Kotlin speaker frames=571 ttf=143.8 ms p50=20.21 p95=20.63 p99=21.45 max=25.88
Rust hang-publish frames=495 ttf=326.0 ms p50=19.99 p95=21.41 p99=22.18 max=26.72
Gated by `-DnestsHangInterop=true`; needs the cargo-built
`hang-publish` sidecar already produced by `interopBuildSidecars`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two production-path bugs surfaced during a full interop sweep
(`./gradlew :nestsClient:jvmTest -DnestsInterop=true` +
`-DnestsHangInterop=true`) plus the toolchain / harness friction that
was hiding them.
Production fixes:
- ReconnectingNestsSpeaker's hot-swap path opens publishers via
`openPublisherForHotSwap` instead of `startBroadcasting`, so the
underlying `MoqLiteNestsSpeaker`'s state machine never made the
Connected -> Broadcasting transition. The reconnect wrapper mirrors
that state, so callers waiting on Broadcasting (the VM,
`connectReconnectingNestsSpeaker` interop tests) hung on Connected
forever. Adds `HotSwappablePublisherSource.reportBroadcasting(isMuted)`,
implemented in `MoqLiteNestsSpeaker`, and the hot-swap pump now
calls it each iteration so a JWT-refreshed session re-enters
Broadcasting too.
- The stale-group filter on listener reconnect assumes monotonic
group lineage across publisher session-restarts (true for
ReconnectingNestsSpeaker, which seeds each new publisher with the
prior `nextSequence`; false for external publishers like
kixelated's hang-publish reference, which mints a fresh state on
every reconnect and restarts at 0). Without compensation, the
watermark from a session-1 max ~18 dropped the entire post-restart
stream from session 2. The collect now resets the watermark on the
first object of a new subscription when its group id arrives well
below the current watermark — a publisher-restart signal that a
relay-cache replay (which carries exactly the prior max) wouldn't
produce.
Test harness fixes — these are what blocked the suites from running
at all on a clean Apple-Silicon box:
- NostrNestsHarness: the cloned `docker-compose-moq.yml` declares no
`depends_on`, so on a cold `up -d` moq-relay raced moq-auth's Node
boot, got `Connection refused` on its JWKS GET, exited with no
retry, and every later QUIC handshake failed with "read loop exited
(socket closed or peer closed)". Gate on moq-auth's /health first,
then idempotent `up -d moq-relay` so the relay always boots against
a live auth sidecar.
- nestsClient/build.gradle.kts: cargo builds inside the hang-interop
task panic under CMake 4.x because `audiopus_sys` / rustls' aws-lc-sys
ship a `CMakeLists.txt` that predates CMake 4's minimum-version
floor. Set `CMAKE_POLICY_VERSION_MINIMUM=3.5` on each cargo Exec.
- JvmOpusEncoder + the hang-interop Test task: opus-java 1.1.1's
bundled `natives/darwin/libopus.dylib` is x86_64-only, so its
in-jar loader reports unsupported on Apple Silicon. Probe a small
set of canonical system libopus locations (`brew install opus` on
macOS, distro paths on linux) and `System.load` by absolute path so
the symbols are in the process when JNA's lazy `Opus.INSTANCE`
init falls back to RTLD_DEFAULT. Gradle also threads
`/opt/homebrew/opt/opus/lib` onto `jna.library.path` for
completeness; both paths are no-ops where the in-jar binary
already loads.
After these:
- :nestsClient:jvmTest -DnestsInterop=true — 312/312, 0 fail
- :nestsClient:jvmTest -DnestsHangInterop=true — 312/312, 0 fail
- quic/interop/run-matrix.sh -s aioquic — 19 passed,
0 failed, 3 unsupported (E/V2/CM are documented amethyst gaps).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- QuicWebTransportFactory: re-throw CancellationException from the bounded
handshake instead of wrapping it in WebTransportException, so a
parent-scope cancellation no longer breaks structured concurrency.
Mirrors the existing handling on the post-CONNECT path.
- NostrNestsHarness: correct two misleading comments — the dev endpoint's
IPv4 guarantee comes from QuicWebTransportFactory's resolve step, not
from `localhost` itself; and DEFAULT_REVISION is documented as tracking
`main` (a known drift risk against the pinned moq-relay), not pinned.
https://claude.ai/code/session_01PoQupfdoKU3ryQwLwTeXeM
The local Docker interop stack builds moq-relay from a `./moq` checkout
that NostrNestsHarness cloned at unpinned `main` (~0.10.25+), while
nostrnests's moq-auth sidecar mints tokens for moq-relay 0.10.10 — the
version nostrnests pins in their own nests-relay/Dockerfile.relay. The
JWKS/JWT-signature handling drifted across those releases, so the relay
rejected every connection with `failed to decode the token` even though
the QUIC/TLS/WebTransport handshake completed cleanly.
Pin DEFAULT_MOQ_REVISION to the moq-relay-v0.10.10 release tag so the
harness builds the relay moq-auth was written against, and fetch tags so
the tag is checkout-able on a pre-existing clone.
https://claude.ai/code/session_01PoQupfdoKU3ryQwLwTeXeM
A stalled/failed QUIC handshake gave no hint whether the socket connected
over IPv4 or fell into the ::1 blackhole. The HandshakeFailed messages
now carry `socket=<addr>:<port> sni=<host>` so the failure itself shows
which address resolvePreferIpv4 picked.
https://claude.ai/code/session_01PoQupfdoKU3ryQwLwTeXeM
After switching the dev harness to `localhost`, the QUIC handshake still
stalled and the relay logged nothing at all — packets weren't reaching
it. `localhost` resolves to ::1 first on most systems, and IPv6 loopback
isn't reliably routable (Docker's IPv6 UDP port-forwarding silently
blackholes), so the UDP socket was connecting into a hole.
QuicWebTransportFactory.connect now resolves the authority host to a
concrete address preferring IPv4 for the UDP socket target, while still
passing the original hostname as the TLS SNI server name — so SNI keeps
matching the server certificate and stays a legal RFC 6066 host_name.
https://claude.ai/code/session_01PoQupfdoKU3ryQwLwTeXeM
Three findings from the local nests interop hang (relay logged "Illegal
SNI extension: ignoring IP address presented as hostname"):
- TlsClientHello sent the connect host verbatim as TLS SNI, including
IP literals. RFC 6066 §3 forbids IP literals in server_name; a strict
relay (rustls) discards the extension, ends up with no SNI to select
its certificate on, and the handshake stalls. Both ClientHello
builders now omit server_name when the host is an IP literal
(new isIpLiteralHostname predicate).
- QuicWebTransportFactory.connect never bounded awaitHandshake() — only
readConnectResponse had a timeout — so a stalled handshake hung
connect() indefinitely. It now fails fast after connectTimeoutMillis
with a message naming the likely cause.
- NostrNestsHarness pointed moqEndpoint at 127.0.0.1, which forced the
IP-literal SNI path against the dev relay. Switched to localhost: a
valid hostname that matches the relay's generated dev cert and still
resolves to loopback.
https://claude.ai/code/session_01PoQupfdoKU3ryQwLwTeXeM
QuicWebTransportFactory threw `:status=0` for every distinct CONNECT
failure mode — relay never answered, relay FIN'd the request bidi with
no H3 response, relay sent a HEADERS frame lacking :status, or the whole
connection was torn down by a CONNECTION_CLOSE. The four cases need
different fixes but produced identical, undiagnosable errors.
readConnectResponse now records bytes/H3-frames seen on the request
stream and reports which of those cases occurred; the thrown exception
also carries a connection-state snapshot (conn.status, closeReason,
closeErrorCode, requestStreamClosed) captured before driver.close(), so
a relay CONNECTION_CLOSE (e.g. H3_SETTINGS_ERROR from a rejected SETTINGS
frame) is distinguishable from a request-stream-only close.
https://claude.ai/code/session_01PoQupfdoKU3ryQwLwTeXeM
SendTraceScenario already swept framesPerGroup and recorded per-frame
send durations + arrival wall-times, but never correlated them — so it
could verify frame delivery but not the send-side latency a listener
perceives. Record each frame's send wall-clock timestamp and derive the
true send→arrival latency (time-to-first-frame + p50/p99/min/max) per
subscriber. With framesPerGroup > 1 this surfaces the batching window
(~framesPerGroup * cadence) as the latency floor.
Add framesPerGroup=10 and =50 (the production default) sweep entries to
both the prod and local-harness suites so the batching-vs-latency
tradeoff curve is visible in one run.
https://claude.ai/code/session_01PoQupfdoKU3ryQwLwTeXeM
On every re-subscribe (listener reconnect or publisher-cycle), the
moq-lite relay re-serves its cached latest group from the first frame.
The wrapper forwarded it straight into the decoder, so in a fully-muted
room the same old clip looped once per reconnect. Track the highest
group sequence delivered by prior subscriptions and drop any group not
strictly newer, mirroring kixelated/hang's Container.Consumer.#run.
https://claude.ai/code/session_01G9h2dzkEj6Y2F1Yr2kCojp
`AudioTrackPlayer` sized the AudioTrack ring at `max(minBuffer * 16,
250 ms)`. The intent (per the kdoc) was ~250 ms of jitter slack with
the device's `getMinBufferSize` as a floor — but the `* 16` multiplier
silently dominated on common devices. A Pixel reports `minBuffer ≈
40 ms` for 48 kHz mono; `× 16 = 640 ms`. Add the 200 ms preroll in
`NestPlayer` and decoded PCM sat in the ring for ~800 ms – 1 s before
the speaker played it.
Two-phone testing confirmed the symptom: the speaking-now ring lit up
~1 s before the listener's audio. The ring fires on
`onLevel(peakAmplitude(pcm))` in `NestPlayer.play()` immediately after
`decoder.decode()` succeeds — before `player.enqueue(pcm)` — so the
ring is real-time and the delay was entirely between `enqueue` and
the speaker. The web (NostrNests) listener uses an `AudioWorklet`
with no equivalent ring buffer, so its audio aligned with the wire.
Drop the `* 16` so the formula matches the intent: `max(minBuffer,
250 ms)`. On devices whose floor is below 250 ms (the common case)
the 250 ms target wins; on devices whose floor is above 250 ms (rare)
we still respect the device-reported minimum. Steady-state ear-to-
speaker delay drops from ~1 s to ~450 ms (250 ms ring + 200 ms preroll).
Removes the four trace points + debug-build auto-enable hook added in
the prior commit on this branch. They served their purpose locally
(confirmed the ~1s startup lag lives between pcm_decoded and
pcm_enqueued, i.e. AudioTrack ring backpressure), and don't need to
land in main.
Adds four trace points on the listener pipeline so an end-to-end JSONL
capture (`adb logcat -s NestsTraceJsonl:D -v raw > nest-trace.jsonl`)
shows wall-clock latency at every stage:
frame_received (MoqLiteSession.drainOneGroup, after trySend)
frame_object_mapped (MoqLiteNestsListener.wrapSubscription map)
pcm_decoded (NestPlayer.play, after decoder.decode)
pcm_enqueued (NestPlayer.play, after player.enqueue)
Each event carries (sub_id, group_seq, ...) or (track_alias, obj_id, ...)
so the same frame can be followed through all four stages. Deltas
isolate which segment owns the observed startup-only ~1s lag:
subscribe_send → first frame_received : relay forward latency
frame_received → pcm_decoded : Opus decode wall-time
pcm_decoded → pcm_enqueued : AudioTrack ring backpressure
pcm_enqueued → next pcm_decoded : decode-loop scheduling
Tracing auto-enables on debug builds when `NestActivity` opens and
disables on `onDestroy` so release apps pay only the field-load
guard inside `NestsTrace.emit`. Capture instructions are inline at
the call site in `NestActivity.onCreate`.
Surface a few code-quality wins surfaced by the simplify pass on the
moq-lite Lite-03/04 audit. All semantic-preserving; tests untouched
in behavior.
- **Derive `MoqLiteSession.version` from `transport.negotiatedSubProtocol`**
instead of carrying it as a separate constructor parameter. The
version was already redundant with the transport's negotiated
ALPN; deriving it eliminates the parameter on
`MoqLiteSession.client(...)` and the `resolveMoqLiteVersion()` /
`moqVersion` plumbing in `connectNestsListener` /
`connectNestsSpeaker`. Tests drive the version via
`FakeWebTransport.pair(negotiatedSubProtocol = …)`, which already
plumbs through to the session's derivation.
- **Extract `MoqLiteSession.rejectSubscribe(bidi, errorCode, reason)`**
helper that writes a `SubscribeDrop` body and `RESET_STREAM`s the
bidi. Both Subscribe-rejection arms (`BROADCAST_DOES_NOT_EXIST`
and `TRACK_DOES_NOT_EXIST`) now share this helper instead of
duplicating the runCatching / write / reset shape. Future drop
codes plug in without growing the duplication.
- **Make `StrippedWtStream.stopSending` non-nullable.** The demux
always wires it (every surfaced stream has a read side), so the
`?:` "defensive" fallbacks in `StrippedWtReadStreamAdapter` and
`StrippedWtBidiStreamAdapter` were dead code. Tightens the
`StrippedWtStream` contract and shrinks the adapters.
- **Extract `SEQ_BITS=23` / `SEQ_MASK=0x007F_FFFF` / `SEQ_MAX`
constants** in `MoqLiteSession.openGroupStream`. The bit-pack
formula now reads as `(trackPriority and 0xFF) shl SEQ_BITS) or
(sequence and SEQ_MASK)` — layout is named, not derived from the
hex literals scattered through the body.
Items deferred (noted but not done):
- `handleInboundBidi` arm extraction — the per-arm variable capture
(`dispatched`, `inboundSub`, `inboundSubPublisher`,
`inboundAnnouncePublisher`) makes a clean refactor harder than
the readability win. Defer until a future feature changes the
dispatch shape and the refactor falls out naturally.
- `Fake{Bidi,Read}Stream` / `ChannelWriteStream` constructor
parameter explosion — the cells-as-nullable-named-params shape
is already readable; the candidate `FakeStreamSignals` struct
would force a left/right-direction flag on the bidi side, which
is uglier than the explicit `myReset` / `peerReset` naming.
- `MoqLiteCodec` `object` → versioned `class` — the
`version: MoqLiteVersion = LITE_03` parameter on each method is
the idiomatic Kotlin shape for an optional-with-default
discriminator; converting to a class would bind state to
instances and complicate the test path that exercises both
versions per-call.
- `announce()` / `probe()` pump abstraction — borderline; the
embedded logging + tracing in `announce()` would have to be
parameterized into the abstraction. Worth revisiting if a third
similar pump appears.
- `MoqLiteSubscribeDropCode` / `MoqLiteStreamCancelCode` sealed-
class refactor — `Long` constants match the wire shape; sealed
types would force conversion at every API boundary
(codec accepts `Long`, transport accepts `Long`).
https://claude.ai/code/session_012TGfo99Ugz7fcCPv85a8Tq
Pre-fix, the M1 priority pack `((trackPriority and 0xFF) shl 24)`
would set bit 31 whenever `trackPriority >= 0x80`, producing a
negative `Int`. `QuicConnectionWriter.sortedByDescending { it.priority }`
sorts via signed `Int.compareTo`, so negative-signed values land
BELOW every positive priority — exactly inverting the intended
"higher trackPriority drains first" ordering.
The default `DEFAULT_TRACK_PRIORITY = 0x80` sits exactly on this
boundary, so this would have hit production the moment a hand-
tuned `trackPriority=0xFF` (highest intended) appeared in any
multi-track scenario.
Reshape the layout to keep bit 31 clear:
bit 31 : 0 (reserved as the sign bit)
bits 30..23 : trackPriority u8 (0..255)
bits 22..0 : sequence low 23 bits (~97 days at 1 grp/sec)
The trackPriority byte still occupies an 8-bit slot, just shifted
one bit lower. Sequence saturation moves from 24 bits (≈ 6 days)
to 23 bits (≈ 97 days) — still ample for the 9-minute JWT refresh
cycle.
Two regression tests:
- The existing `publisher_packs_trackPriority_and_sequence_into_setPriority_value`
is updated to assert the new bit layout AND that the encoded
value is non-negative.
- New `publisher_priority_pack_keeps_top_trackPriority_above_lower_trackPriority`
is the direct guard against the bug: `trackPriority=0xFF`
must encode HIGHER than `trackPriority=0x7F`. Pre-fix this
test fails with `0xFF` encoding to a negative Int below the
positive `0x7F` encoding; post-fix both are positive and
correctly ordered.
Surfaced by the merge-prep security review of the moq-lite
Lite-03/04 audit branch — agent flagged it as "QoS/correctness
not security, excluded by DoS rule" but it's a real bug worth
fixing before merge.
Audit doc updated: bit layout in `nestsClient/plans/2026-05-09-moq-lite-rfc-compliance.md`
(M1 + L4 entries).
https://claude.ai/code/session_012TGfo99Ugz7fcCPv85a8Tq
Adds Fix#6 (L2 — SubscribeOk narrowing), Fix#7 (L3 —
subscriber-driven Probe API), and Fix#8 (L1 — version-aware
Lite-03/04 codec + ALPN negotiation) to the compliance audit
document. Updates the gap matrix (L1/L2/L3 → ✅) and the TL;DR
(now reads "nine shipped fixes + M6 closed; every gap is now
resolved, no items remain deferred").
The "what's deliberately deferred" section now lists only the
external follow-up — a `kixelated/moq` feature request
suggesting per-deployment tuning of moq-rs's per-subscriber
forward-queue starvation behavior. That's an upstream-relay
concern, not a moq-lite gap.
Updates the audio-rooms completion-plan pointer to reflect the
shipped count and the absence of remaining deferrals. Audit doc
title acknowledges Lite-04 alongside Lite-03 since the codec
now speaks both.
https://claude.ai/code/session_012TGfo99Ugz7fcCPv85a8Tq
Lite-03 audit L1: completes the moq-lite version surface so the
session speaks the wire-format version the server picks via
WebTransport ALPN negotiation. Pre-fix the codec hardcoded Lite-03;
advertising `moq-lite-04` was a footgun because a Lite-04-preferring
relay would desync on the very first Announce exchange.
Verified diff between Lite-03 and Lite-04 by WebFetching kixelated's
Rust reference (`rs/moq-lite/src/lite/{announce,subscribe,probe}.rs`,
`rs/moq-lite/src/version.rs`, `rs/moq-lite/src/model/origin.rs`):
exactly three fields differ; everything else is byte-identical.
- `Announce.hops`: Lite-03 wire = single varint count (the spec
explicitly fills with `Origin::UNKNOWN` placeholders on decode);
Lite-04 wire = `varint(count) + count × varint(originId)` (the
`OriginList`). MAX_HOPS = 32.
- `AnnouncePlease.excludeHop`: Lite-03 absent; Lite-04 single
varint after `prefix`. Sentinel `0` = no exclusion.
- `Probe.rtt`: Lite-03 absent; Lite-04 single varint after
`bitrate`. Sentinel `0` = unknown (decoded as null). Outgoing
`Some(0)` is clamped to `Some(1)` to avoid colliding with the
sentinel — mirrors kixelated's `encode_msg` clamp.
New surface:
- `MoqLiteVersion` enum (LITE_03, LITE_04) with `fromAlpn` lookup.
- `MoqLiteCodec.{encode,decode}{AnnouncePlease,Announce,Probe}`
take `version: MoqLiteVersion = LITE_03` parameter, branch on
it. Default LITE_03 keeps existing call sites compiling.
- `MoqLiteSession.client(transport, scope, version)` carries the
version through every codec invocation.
- `MoqLiteAnnouncePlease.excludeHop: Long = 0L` (default).
- `MoqLiteAnnounce.hops: List<Long>` (was `Long`) — list of
origin IDs, bounded to MAX_HOPS=32. Existing call sites
migrate `hops = 0L` → `hops = emptyList()`, `hops = 7L` →
`hops = List(7) { 0L }`.
- `MoqLiteProbe.rtt: Long? = null` (default).
Negotiation:
- `WebTransportSession.negotiatedSubProtocol: String?` exposes
the server's `wt-protocol` selection. `QuicWebTransportFactory`
parses the response HEADERS, extracts the SF-string from
`wt-protocol`, and threads it into `QuicWebTransportSession`.
`FakeWebTransport.pair(negotiatedSubProtocol = …)` lets tests
drive the same path.
- Default advertised list now `[moq-lite-04, moq-lite-03]` (was
`[moq-lite-03]`). Lite-04 sits first to match kixelated's
preference; servers that don't support it fall back to
Lite-03 cleanly.
- `connectNestsListener` / `connectNestsSpeaker` resolve the
version via `resolveMoqLiteVersion(webTransport.negotiatedSubProtocol)`
and pass it to `MoqLiteSession.client(...)`. Falls back to
Lite-03 when the server doesn't echo `wt-protocol` (older
deployments) or echoes something unrecognised (forward-compat).
- New `MOQ_LITE_04_VERSION = 0x6D71_6C04L` synthetic version
code; `versionCode(version)` mapping function.
Regression tests (all green):
- `MoqLiteCodecTest.announcePlease_lite04_round_trips_excludeHop`
- `MoqLiteCodecTest.announcePlease_lite03_omits_excludeHop_on_wire`
- `MoqLiteCodecTest.announce_lite04_round_trips_full_origin_list`
- `MoqLiteCodecTest.announce_lite03_drops_origin_ids_keeps_count`
- `MoqLiteCodecTest.announce_decoder_rejects_oversize_hop_count`
(MAX_HOPS bounds check)
- `MoqLiteCodecTest.probe_lite04_round_trips_rtt`
- `MoqLiteCodecTest.probe_lite04_clamps_some_zero_to_one_to_avoid_unknown_sentinel`
- `MoqLiteCodecTest.probe_lite04_decodes_zero_rtt_as_null`
- `MoqLiteCodecTest.probe_lite03_wire_omits_rtt`
- `MoqLiteSessionTest.lite04_announce_round_trips_full_origin_list_through_session`
(end-to-end Lite-04 session)
Audit doc: nestsClient/plans/2026-05-09-moq-lite-rfc-compliance.md (L1).
https://claude.ai/code/session_012TGfo99Ugz7fcCPv85a8Tq
Lite-03 audit L3: completes the subscriber-side Probe surface to
mirror kixelated's `Subscriber::run_probe_stream`
(`rs/moq-lite/src/lite/subscriber.rs`).
`MoqLiteSession.probe()` opens a bidi, writes
`ControlType::Probe` (varint 4), and returns a `MoqLiteProbeHandle`
whose `updates` flow yields each size-prefixed `MoqLiteProbe`
message the publisher pushes. The handle's `close()` FINs the
bidi and cancels the pump.
`updates` is a `MutableSharedFlow(replay=8)` so a collector that
attaches after the publisher's first emit doesn't miss it
(matches the same shape the announce-watch uses).
No production consumer for the API today — Amethyst's nests
listener doesn't run ABR on a fixed-rate Opus encoder. The API
exists to round out the moq-lite Lite-03 surface: a diagnostic
tool can now read a publisher's bitrate without subscribing to
its data track.
The publisher-side handler (existing) was already correct: it
writes one `MoqLiteProbe` (32 kbps Opus voice hint) and FINs.
Regression test:
`MoqLiteSessionTest.subscriber_probe_writes_control_type_and_decodes_publisher_bitrate`.
Audit doc: nestsClient/plans/2026-05-09-moq-lite-rfc-compliance.md (L3).
https://claude.ai/code/session_012TGfo99Ugz7fcCPv85a8Tq
Lite-03 audit L2: when accepting a SUBSCRIBE, the publisher MAY
narrow the subscriber's `startGroup` / `endGroup` request bounds.
Pre-fix we always echoed `null/null`, which lost the diagnostic
"which group am I about to start sending?" information — particularly
useful for hot-swap continuations where the publisher's
[MoqLitePublisherHandle.nextSequence] is non-zero (the seeded
`startSequence` from the previous moq-lite session).
The fix narrows `startGroup` to `targetPublisher.nextSequence`. The
subscriber decodes this as "the next group on this subscription
will be sequence N" and can log / surface the join-point. `endGroup`
stays null because live audio rooms have no end in sight; the
subscriber's request bound is honoured implicitly when the publisher
closes.
Regression test:
`MoqLiteSessionTest.publisher_subscribeOk_narrows_startGroup_to_next_sequence`.
Audit doc: nestsClient/plans/2026-05-09-moq-lite-rfc-compliance.md (L2).
https://claude.ai/code/session_012TGfo99Ugz7fcCPv85a8Tq
Adds Fix#5 (M2 + M3 — STOP_SENDING for single-group cancel +
RESET_STREAM with typed code on Drop replies) to the compliance
audit doc. The two items shipped together because the plumbing is
shared — extending `:quic`'s StrippedWtStream with reset/stopSending
closures, extending the WebTransport interfaces, and routing through
all adapters was the prerequisite for both.
Updates the gap matrix (M2 + M3 → ✅), the TL;DR (now reads "six
shipped fixes + M6 closed; no 🟡 items remain open"), and the
deferred-items list (only L1 / L2 / L3 remain, all 🟦 with explicit
rationale).
Updates the audio-rooms completion-plan pointer with the new
shipped count.
https://claude.ai/code/session_012TGfo99Ugz7fcCPv85a8Tq
Lite-03 audit M2: when a group's uni stream arrives for a
`subscribeId` whose subscription has already been removed (typical
case: listener canceled / unsubscribed before the publisher
observed it), the listener MUST signal the publisher to abandon
in-flight retransmits via `STOP_SENDING(applicationErrorCode)` on
that uni stream. Pre-fix, `drainOneGroup` silently dropped every
frame (`droppedNoSub++`) and let the publisher keep pushing until
natural FIN — wasted bandwidth on bytes the listener would discard,
plus relay queue pressure on the per-subscriber forward pipeline
that already sits behind the production stream-cliff.
Wire the latched `stopSending(MoqLiteStreamCancelCode.SUBSCRIPTION_GONE)`
on the first frame that observes `sub == null`. Latched (one-shot)
so concurrent frames in the same drain don't re-fire — RFC 9000
§3.5 says subsequent STOP_SENDINGs are ignored anyway, but it saves
the syscall and keeps the trace clean.
New error-code constant `MoqLiteStreamCancelCode.SUBSCRIPTION_GONE
= 0x10L` lives next to `MoqLiteSubscribeDropCode` in
`MoqLiteMessages.kt`. moq-lite leaves application error codes
undefined; we follow the same "small non-zero varint" convention.
Test seam: `ChannelWriteStream` is now a public class (was
private) with a `peerStopSendingCode` accessor, so a test that
holds the writer side (`serverSide.openUniStream() as
ChannelWriteStream`) can assert the listener's stopSending code
without racing for the peer-side `FakeReadStream` reference.
Regression test:
`MoqLiteSessionTest.listener_stopSending_group_uni_when_subscription_already_canceled`.
Audit doc: nestsClient/plans/2026-05-09-moq-lite-rfc-compliance.md (M2).
https://claude.ai/code/session_012TGfo99Ugz7fcCPv85a8Tq
moq-lite Lite-03 conveys application-level errors on any stream via
`RESET_STREAM(application_error_code u32)` (audit M3). Pre-fix, our
publisher's two SubscribeDrop reply paths
(`BROADCAST_DOES_NOT_EXIST`, `TRACK_DOES_NOT_EXIST`) wrote the Drop
body and then `bidi.finish()` — a graceful FIN — which overlapped
with "publisher gracefully shut down." A subscriber watching only
the QUIC layer (no body decode) couldn't tell rejection from
shutdown.
Replace the trailing `bidi.finish()` with `bidi.reset(errorCode)`
in both Drop paths. The errorCode matches the body's `errorCode`
field so a peer that decodes the Drop sees the same number as one
that only sees the RESET_STREAM frame. The plumbing landed in the
prior commit (`feat(quic,nestsclient): plumb RESET_STREAM +
STOP_SENDING through WebTransport`).
Tests: existing `publisher_replies_subscribeDrop_when_*` regressions
gain an additional `lastPeerResetCode` assertion via
`FakeBidiStream`, locking the typed-error contract.
Audit doc: nestsClient/plans/2026-05-09-moq-lite-rfc-compliance.md (M3).
https://claude.ai/code/session_012TGfo99Ugz7fcCPv85a8Tq
Extends the WebTransport interface and its three implementations
(QUIC-backed, in-memory fake, peer-stripped demux) with typed
error-code cancel paths so moq-lite Lite-03's M2/M3 audit items can
land.
QUIC layer (`:quic`):
- `StrippedWtStream` gains optional `reset` and `stopSending`
closures, wired by `WtPeerStreamDemux.emitStripped` through
`QuicStream.resetStream(errorCode)` /
`QuicStream.stopSending(errorCode)` + `driver.wakeup()` so the
frames actually leave the connection. `reset` is null on
peer-initiated uni streams (no send half on our side);
`stopSending` is unconditional (every surfaced stream has a
read side).
WebTransport interface (`:nestsClient`):
- `WebTransportReadStream.stopSending(errorCode: Long)` —
suspending; first-call-wins.
- `WebTransportWriteStream.reset(errorCode: Long)` —
suspending; first-call-wins. Distinct from the existing
graceful `finish()` / FIN.
Adapters:
- `QuicBidiStreamAdapter`, `QuicReadStreamAdapter`,
`QuicUniWriteStreamAdapter` route directly to the underlying
`QuicStream` API.
- `StrippedWtReadStreamAdapter` /
`StrippedWtBidiStreamAdapter` route through the demux's
closures; defensive `?:` fallbacks defend against a future
demux bug that forgets to wire a closure.
- `FakeBidiStream`, `FakeReadStream`, `ChannelWriteStream` (the
in-memory test transport) record reset / stopSending codes
in shared `AtomicLong` cells so tests can assert "the peer
reset with code X" / "the listener stopSending with code Y"
via new `lastResetCode` / `lastStopSendingCode` /
`lastPeerResetCode` / `lastPeerStopSendingCode` properties.
Sentinel `NO_CODE = Long.MIN_VALUE` distinguishes "not called"
from a legitimate code 0.
Pure plumbing — no moq-lite call sites use the new methods yet.
Subsequent commits wire M3 (publisher Drop reply uses
reset(errorCode)) and M2 (listener group-cancel uses
stopSending(errorCode)).
Audit doc: nestsClient/plans/2026-05-09-moq-lite-rfc-compliance.md (M2 + M3).
https://claude.ai/code/session_012TGfo99Ugz7fcCPv85a8Tq
Adds Fix#3 (M1 — trackPriority + sequence bit-pack matching kixelated
PriorityHandle), Fix#4 (L5 — fresh publisher list at dispatch
time), and the M6 closure (verified via WebFetch that Goaway has no
body schema in moq-lite Lite-03 — `rs/moq-lite/src/lite/{stream,
client}.rs`; our recognise/log/FIN handler is canonical) to the
compliance audit document.
Also marks L4 closed (subsumed by M1 — the saturating cast guard
collapsed into the per-track high byte logic).
Updates the audio-rooms completion-plan pointer with the new shipped
count: four publisher-side spec tightenings + one closure (M6) +
five 🟡 / 🟦 items explicitly deferred with rationale.
Remaining deferrals: M2 + M3 (need WebTransport/`:quic` interface
extension, locked per audit prompt), L1 (Lite-04 codec rewrite, no
relay forces it), L2 + L3 (no consumer for the API).
https://claude.ai/code/session_012TGfo99Ugz7fcCPv85a8Tq
`handleInboundBidi` previously snapshotted the publisher list at the
TOP of the function (= bidi-arrival time) and used the snapshot for
the rest of the bidi's lifetime. A publisher registered between
bidi-open and the first inbound chunk would miss the dispatcher's
view, even though the publisher was already live in `activePublishers`
by the time we needed to dispatch.
Practical impact today: zero — both nests publishers (`audio/data`
and `catalog.json`) register from `MoqLiteNestsSpeaker.startBroadcasting`
before the relay's SUBSCRIBE bidi for either track lands. The
~few-ms gap is below the network round-trip floor.
But the contract is narrower if we read the list at first-byte time:
we then see every publisher that was registered up to the moment we
needed to make a routing decision. Closes the L5 row of the audit doc.
Move the publishers fetch from the function-top to inside the
`if (!dispatched)` block, after the control-type byte has been read.
On empty publisher list (the case we previously short-circuited at
function-top), we now FIN cleanly so the peer's wait resolves
instead of hanging on an idle bidi.
Audit doc: nestsClient/plans/2026-05-09-moq-lite-rfc-compliance.md (L5).
https://claude.ai/code/session_012TGfo99Ugz7fcCPv85a8Tq
Per moq-lite Lite-03 (kixelated/moq `rs/moq-lite/src/lite/priority.rs`),
`Publisher::serve_group` calls `priority.insert(track.priority,
sequence)` and feeds the resulting position into `stream.set_priority`.
Priority sorts first by `track.priority u8` (higher track = drains
ahead under congestion), then by group `sequence` within a track
(newer = drains ahead).
Pre-fix we passed raw `sequence.toInt()` directly to setPriority,
ignoring the per-track byte. For our single-Opus-track production case
this was unobservable — newer-first ordering held by sequence
monotonicity — but a future multi-track broadcast (audio + companion
catalog / status track) would have starved the lower-rate track the
moment audio's outbound queue got congested.
Fix: add `trackPriority: Int = DEFAULT_TRACK_PRIORITY` parameter to
`MoqLiteSession.publish()`, store on PublisherStateImpl, and bit-pack
in `openGroupStream`:
bits 31..24 trackPriority u8 (0..255)
bits 23..0 sequence low 24 bits
The 24-bit sequence window is ample (≈ 6 days at 1 group/sec, beyond
which all newer groups within a single track tie — but they still
beat older groups of any LOWER-priority track via the top byte).
Production sessions cycle on JWT refresh every 9 min, so the wrap is
defensive only.
`DEFAULT_TRACK_PRIORITY = 0x80` matches the existing subscriber-side
DEFAULT_PRIORITY midpoint, so all existing call sites keep their
prior behavior.
Test seam: `FakeWebTransport.openUniStream` now records the most-
recent `setPriority` value via a shared `AtomicInteger` cell that the
peer-side `FakeReadStream` exposes as `lastSetPriority`. Lets the new
regression test verify the bit-pack formula on the actual peer-side
read stream rather than peeking into private state.
Regression test:
`MoqLiteSessionTest.publisher_packs_trackPriority_and_sequence_into_setPriority_value`.
Audit doc: nestsClient/plans/2026-05-09-moq-lite-rfc-compliance.md (M1).
https://claude.ai/code/session_012TGfo99Ugz7fcCPv85a8Tq
Records the cross-spec compliance audit of the moq-lite Lite-03
implementation against kixelated's reference Rust impl
(github.com/kixelated/moq, rs/moq-lite/). Methodology mirrors the
recent QUIC RFC review: walked every Kotlin source under
`moq/lite/`, traced data flow speaker → relay → listener, and
cross-referenced each on-wire message + control byte against the
Rust reference.
Outcome: no 🔴 wire-incompatibilities found. Every codec primitive —
control type discriminators, AnnouncePlease, Announce, Subscribe
(incl. the `Option<u64>` off-by-one and `Duration` millis-as-varint
conventions), SubscribeOk/Drop with the type-outside-size-prefix
framing peculiar to Lite-03, GroupHeader, Probe — matches byte-for-
byte.
Six 🟡 (spec-loose / future-fragile) and four 🟦 (diagnostic /
observability) gaps documented:
- M1 stream priority parity vs kixelated's PriorityHandle
(single-track Opus impact: invisible)
- M2 STOP_SENDING for single-group cancel not exposed
- M3 RESET_STREAM with Error::to_code not used
- M4 AnnouncePlease prefix-mismatch falsely emits Active — fixed
in 6c2d7efc
- M5 inbound Subscribe doesn't validate broadcast field — fixed
in 3dcee2a4
- M6 Goaway body / migration handler
Also references the new audit from `2026-04-26-audio-rooms-completion.md`
so future readers find it via the completion-plan pointer.
https://claude.ai/code/session_012TGfo99Ugz7fcCPv85a8Tq
The publisher-side dispatcher in handleInboundBidi previously matched
inbound Subscribe bidis on `track` only, never checking whether the
requested `broadcast` field matched the suffix our publisher claimed
on this session. A peer (or buggy relay) could open a Subscribe under
broadcast="otherPubkey" track="audio/data" and we would route OUR
audio frames to them — the dispatcher only filtered on track.
Production never bit because moq-rs's relay routes Subscribe messages
to the specific publisher whose broadcast matches the requested path
upstream of us, so an off-target broadcast never landed on our
inbound bidi pump. But a peer-to-peer connection without a relay
in between would have been able to siphon our audio under any
broadcast string they invented.
The fix: validate `sub.broadcast == publisher.suffix` (both already
path-normalised by the codec) before track matching. On mismatch,
reply `SubscribeDrop(errorCode=BROADCAST_DOES_NOT_EXIST,
reasonPhrase="<requested> not published on this session
(we publish <ours>)")` and FIN. New error code
`MoqLiteSubscribeDropCode.BROADCAST_DOES_NOT_EXIST = 0x05L` mirrors
the IETF MoQ-transport `TRACK_NAMESPACE_DOES_NOT_EXIST` semantic so
a watcher can tell apart "wrong room" from "wrong rendition".
Regression test:
`MoqLiteSessionTest.publisher_replies_subscribeDrop_when_broadcast_does_not_match`.
Audit doc: nestsClient/plans/2026-05-09-moq-lite-rfc-compliance.md (M5).
https://claude.ai/code/session_012TGfo99Ugz7fcCPv85a8Tq
Per moq-lite Lite-03 (kixelated/moq `rs/moq-lite/src/lite/announce.rs`),
a publisher MUST only emit `Announce(Active)` for broadcasts whose path
starts with the requested AnnouncePlease prefix. Pre-fix, when
`MoqLitePath.stripPrefix(please.prefix, ourSuffix)` returned `null`, we
fell through to `?: announcePublisher.suffix` and emitted Active under
our full suffix anyway, falsely advertising the broadcast under a
prefix the subscriber didn't request.
Production never bit because the relay always opens its announce bidi
to us with `prefix=""` (which always matches), but a future peer-to-
peer or namespace-scoped subscriber would have observed a ghost
broadcast under whatever prefix they asked about.
The fix: if `stripPrefix` returns `null`, FIN the bidi cleanly without
writing any Announce body. The subscriber sees an empty announce
stream and moves on.
Regression test:
`MoqLiteSessionTest.publisher_skips_announce_when_announce_please_prefix_does_not_match`.
Audit doc: nestsClient/plans/2026-05-09-moq-lite-rfc-compliance.md (M4).
https://claude.ai/code/session_012TGfo99Ugz7fcCPv85a8Tq
Removes the `hang-interop` + `browser-interop` jobs from
`build.yml` (added in commit `21947bc5` after a 10/10 stability
sweep × 22 tests = 220/220 hard-pass). Cold-cache cost is
~10 min hang + ~13 min browser; warm-cache critical-path add
is ~6 min via the parallel browser job. Most PRs don't touch
audio / MoQ / QUIC, so paying that cost on every PR is
net-negative for the change-pattern the repo sees.
Adds `nestsClient/tests/README.md` covering:
- when to run (changes to nip53 / moq-lite session / audio
pipeline / MoqLiteNests* / ReconnectingNests* / :quic /
the sidecars themselves);
- quick-start gradle commands for hang-only, browser-only,
combined;
- prerequisites + first-run cache costs;
- all the configuration knobs (incl. the
`-DnestsHangInteropTraceRelay=true` trace-capture switch
added during the routing investigation);
- known limitations (hot-swap browser soft-pass,
framesPerGroup pin-vs-prod gap, I7 cycle 2 truncation);
- a 4-step debug recipe for triaging a flaking scenario.
Plan updates:
- `2026-05-07-t16-closure-roadmap.md` — Priority 3 marked
⏸ DEFERRED instead of ✅ CLOSED, pointing at the new README.
- `2026-05-07-cross-stack-interop-ci-gating.md` — status
changed to ⏸ DEFERRED; YAML shape preserved verbatim in the
plan for the next revisit.
- `2026-05-06-cross-stack-interop-test-results.md`,
`2026-05-06-cross-stack-interop-test-gap-matrix.md` — CI
integration § rewritten to "manual-run only" + README link.
The trace-capture instrumentation in `NativeMoqRelayHarness`
stays in place; it's useful for future flake triage even
without CI.
Self-audit of commits `d7f87971` (trace capture) and `f8dc9c59`
(hot-swap soft-pass revert) caught four issues; this commit
addresses them.
`NativeMoqRelayHarness.kt`:
- `ProcessOutputDrainer.start`: tolerate `bufferedWriter()`
failures so a misconfigured trace-log dir (parent gone, disk
full, etc.) doesn't kill the drain thread and deadlock the
relay subprocess on a full stdout pipe (~64 KB Linux pipe
buffer). Fall back to ring-only capture and System.err-warn,
matching the pattern the relay-startup error handler already
uses.
- Hoist `Regex("[^A-Za-z0-9._-]")` to `tagSanitiser` so we don't
recompile per relay boot.
- Rename `LOG_TIMESTAMP_FMT` → `logTimestampFmt` (it's a runtime
`val`, not a `const val`; existing convention is camelCase for
runtime, SCREAMING_SNAKE for compile-time constants like
`PORT_READY_TIMEOUT_MS`).
`BrowserInteropTest.kt`:
- `chromium_listener_speaker_hot_swap_does_not_crash`: prune the
`pcm.size <= warmupSamples` early-return + the trailing comment
about the skipped FFT. After the soft-pass revert the
post-warmup branch had no assertions, so the early-return was
dead code. Reduce to a single decoderErrors assertion + kdoc
spelling out the soft-pass and pointing at the hang-tier T12
counterpart.
- Update the kdoc to reflect what the test ACTUALLY asserts (it
used to claim FFT-peak coverage that no longer applies).
Other audit findings deferred (not real bugs, low priority):
- `ConcurrentLinkedQueue.size()` O(n) per line in the drainer.
- Per-line `writer.flush()` syscall — kept intentionally for
hung-test post-mortem; documented inline.
- BrowserInteropTest at 1140+ lines could split helpers — pre-
existing situation, not a regression.
Follow-up to commit `029329af`: a verification sweep (5×) showed
the `pcm.size > warmupSamples` hard floor on
`chromium_listener_speaker_hot_swap_does_not_crash` flakes 3/5 —
empirical capture varies 2880–7680 samples (= 60–160 ms TOTAL),
straddling the 4800-sample warmup threshold. The browser-side
@moq/lite 0.2.x re-attach behaviour across `Active::Ended →
Active` is fundamentally unreliable; ANY hard floor that
excludes the regression mode ("swap killed the WT session
entirely") fail-flakes because the steady state itself sits
right at that threshold.
Per `2026-05-07-tighten-cross-stack-assertions.md`'s Risk § option
(a) — "If any scenario fail-flakes after tightening, [...] revert
the tightening on that scenario" — this commit reverts the
hot-swap floor to a documented soft-pass that prints to
System.err when triggered. T12 (group sequence carry across
hot-swaps) is still asserted by the hang-tier counterpart
(`speaker_hot_swap_does_not_crash` in `HangInteropTest`), which
hard-asserts the full post-swap window decodes the 440 Hz peak.
The deferred "browser hot-swap re-attach" follow-up in
`2026-05-06-cross-stack-interop-test-results.md` captures the
underlying @moq/lite client work that would let this scenario
become a hard-floor test in the future.
5/5 sweep × 22 tests = 110/110 hard-pass post-recalibration
(commits `04be38ad` + `029329af`). Marks
`2026-05-07-tighten-cross-stack-assertions.md` and the roadmap's
Priority 2 closed; documents the per-scenario floors that landed,
including the one weaker-than-specced floor on the browser
hot-swap (deferred follow-up). Priority 3 (CI gating) is now
unblocked.
The first 5× sweep after Priority 2's tightening (commit `04be38ad`)
exposed two scenarios where my chosen thresholds didn't match the
post-merge browser path:
1. **`chromium_listener_mid_broadcast_mute_shortens_pcm`** —
the plan recommended tightening the no-mute upper bound from
5.5 s to 5.0 s. Empirical post-`:quic`-merge steady state is
~5.1–5.2 s (Chromium AudioDecoder ramp-up + harness window
padding); 5.0 s tripped 5/5 sweeps. Reverted to 5.5 s — still
excludes the 6 s "push embedded silence instead of FIN"
regression mode.
2. **`chromium_listener_speaker_hot_swap_does_not_crash`** —
the plan recommended a 0.5 s floor. Empirical post-merge sample
count is ~100–160 ms (warmup window only). Looks like
Chromium's `@moq/lite` 0.2.x client tears down its catalog/audio
subscriptions when it sees `Announce::Ended → Active` in rapid
succession instead of re-attaching. Tracked as a deferred
follow-up in `2026-05-06-cross-stack-interop-test-results.md`.
Replaced the 0.5 s floor with `pcm.size > warmupSamples`
(catches "swap killed the WT session entirely"). The hang-tier
counterpart (`speaker_hot_swap_does_not_crash`) hard-asserts the
full post-swap window decodes the 440 Hz peak, so T12
protection is intact via the hang tier; the browser tier here
only asserts the WT session survived the swap. FFT assertion
removed because the captured window is too short post-merge for
the FFT to resolve a 440 Hz peak with halfWindowHz=5.
Per the plan's Risk § option (b): widen the threshold ONLY if the
new value still excludes the regression mode the test was designed
to catch.
T16 closure-roadmap Priority 2 (`2026-05-07-tighten-cross-stack-assertions.md`).
Replaces every `if (pcm.size <= warmupSamples) return@runBlocking`
short-circuit in `BrowserInteropTest` with a sample-count
`assertTrue` floor:
- I2 late-join: `≥ 1.5 s after warmup` (was vacuous-pass on cold-launch flake)
- I3 mute-window: `≥ 2.5 s` lower bound + tightened `< 5.0 s` upper (was `< 5.5 s`)
- I4 stereo: `≥ 1 s × 2 channels` (= sample-rate × 2 floats)
- I5 hot-swap: `≥ 0.5 s after warmup`
- I9 packet-loss: `≥ 0.5 s after warmup`
- I14 decoder-no-errors: `decoderOutputs ≥ 4` (3 warmup + ≥ 1 audio)
- Browser-publish baseline + reconnect: `runBrowserPublishKotlinListen`
helper hard-asserts on listener side instead of System.err-printing
and returning early.
These were soft-passes because the pre-merge `:quic` post-handshake
bidi-drop bug produced 0-frame outcomes ~50 % of the time, and we
didn't want to fail-flake the suite while the actor was still
unidentified. Trace capture in commit `b2a42d9a` pinned that
actor on `:quic`; merging `origin/main` (commit `8f8251a5`) closed
it; commit `eea746a6` documented the closure.
Sweep verification of the hardened assertions is pending — the
hardening commit lands first so the verification sweep runs
against committed state.
Gap matrix updated to reflect the post-merge stability + hard
floors landing.
club.minnced:opus-java doesn't ship natives for darwin-aarch64, so the
sine-440 round-trip test failed on Apple Silicon dev machines and blocked
the pre-push hook. Catch the IllegalStateException from encoder/decoder
construction and use JUnit Assume to skip; Linux x86_64 CI keeps coverage.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Step 1 of `2026-05-07-moq-relay-routing-investigation.md` ran a
5× sweep on `HangInteropTest` with per-test moq-relay trace
capture. Failure rate: 3/5, all in
`late_join_listener_still_decodes_tail`. Cross-referencing the
relay trace (`encoding self=Subscribe …catalog.json` on conn{id=0}
at T+2.07 s) with the speaker's `Log.d("NestTx")` lines (only
`ANNOUNCE inbound prefix=''` at T=0; NO `SUBSCRIBE inbound id=0`
in the failing window) shows the relay correctly forwards the
upstream SUBSCRIBE — the bidi just never reaches
`MoqLiteSession.handleInboundBidi`. So the prior plan's
"moq-relay 0.10.x per-broadcast subscribe-routing race"
hypothesis is wrong.
Re-scoped Priority 1 of the closure roadmap to point at
`:quic`'s peer-bidi surfacing path instead. The actual bug is
between QUIC's bidi-accept and `incomingBidiStreams()` flow.
Spotless-formatted Kotlin imports.
Trace artefacts preserved under
`nestsClient/plans/artefacts/2026-05-07-routing-race-disproven/`
so the next agent (QUIC owner) doesn't have to re-run the sweep.
Walks through `moq-relay/src/connection.rs`,
`moq-lite/src/lite/publisher.rs`, `moq-relay/src/cluster.rs` and
`moq-lite/src/model/{origin,broadcast}.rs` to refine the routing-race
hypotheses (`H1` / `H1b` / `H1c`). Documents that even main HEAD
`bdda6bd1` keeps `recv_subscribe`'s synchronous `consume_broadcast`
lookup; commit `bea9b3a` only added `wait_for_broadcast` as an opt-in
alternative and an explicit TODO acknowledging the race in
`moq-relay/src/web.rs:325`. So Step 4 (version bump past 0.10.25)
won't yield a fix — the most viable upstream change would convert
`recv_subscribe` to `origin.wait_for_broadcast(path).await` with a
bounded deadline. Concrete trace from Step 1 still needed to pick
between the surviving hypotheses.
`NativeMoqRelayHarness` gains a `testTag` parameter on `shared` /
`resetShared` and writes the relay subprocess's combined stdout/stderr
to `nestsClient/build/relay-logs/<tag>-<seq>-<ts>.log` whenever
`-DnestsHangInteropTraceRelay=true` is set. The subprocess runs with
`RUST_LOG=info,moq_relay=trace,moq_lite=trace,moq_native=debug` so the
per-broadcast subscribe-routing path investigated in
`nestsClient/plans/2026-05-07-moq-relay-routing-investigation.md`
(Step 1) becomes visible.
`HangInteropTest` and `BrowserInteropTest` now expose a JUnit 4
`TestName` rule and forward the method name into `resetShared`, so
each scenario's per-method log is easy to locate by name when
cross-referencing with the speaker-side `Log.d("NestTx")` lines
already captured in JUnit XML's `<system-err>`.
8 plan files updated to reflect what's actually shipped vs. what
each doc said before:
- 2026-05-06-cross-stack-interop-test.md: 'Spec — ready to
implement' → 'Implemented and merged' with pointers to results,
gap matrix, and closure roadmap.
- 2026-05-06-cross-stack-interop-test-results.md: scenario
inventory updated to show all branches merged into
claude/cross-stack-interop-test-XAbYB; suite-flake caveats
flagged with cross-refs to the routing investigation; file
inventory now lists BrowserInteropTest + PlaywrightDriver +
the moved nestsClient/tests/browser-interop/ tree (no more 'in
sister branches not yet merged' section).
- 2026-05-06-cross-stack-interop-test-gap-matrix.md: T8/T10/T11/
T12/T13 all show hang ✅ + browser ✅; I13/I14 'NOT YET LANDED'
callouts removed; the 'in sister branch' qualifier on T13 / I7
/ I6 dropped; coverage-holes section replaced with caveat
pointers to the routing investigation.
- 2026-05-06-i4-stereo-cross-stack-scenario.md: 'Spec — ready for
implementation' → 'Landed' with PR #2755 + the three test
scenarios that ship the assertion.
- 2026-05-06-phase4-browser-harness.md: 'Spec — ready for
implementation' → 'Landed', with notes on how the implementation
diverged (used Container.Legacy.Consumer not @moq/watch; cert
pinning via serverCertificateHashes; I2/I3/I4/I13/I14 originally
deferred but later landed).
- 2026-05-06-phase4-browser-harness-results.md: status updated to
reflect 4.D CI removed per maintainer ask; layout note for
the nestsClient/tests/browser-interop/ relocation; full
scenario list reconciled with results doc.
- 2026-05-07-late-join-catalog-flake-investigation.md: status
updated to mention 00f6cba31 + 207057374 were reverted, and
link to 2026-05-07-moq-relay-routing-investigation.md as the
action plan. Adds note that the flake also affects four
browser-tier scenarios after Browser I7 landed.
- 2026-05-01-quic-stream-cliff-investigation.md: adds an HCgOY-
override banner — recommendation 2 (framesPerGroup = 5) was
superseded 4 days later by HCgOY field tests landing the prod
default at 50. Cross-refs the reconciliation + production-
rerun plans. Open follow-up #3 ('reset to 1') updated with
the same context.
No code or test changes.
Mirrors the existing nestsClient/tests/hang-interop/ layout — all
T16 cross-stack interop test infrastructure (Rust sidecars + bun
browser harness) now lives under nestsClient/tests/.
Updates:
- nestsClient/build.gradle.kts: browserInteropDir path.
- PlaywrightDriver.kt + BrowserInteropTest.kt: kdoc comments.
- All plan docs in nestsClient/plans/ that referenced the old
path.
Compiles clean post-move. No CI changes (those jobs are
intentionally not wired per 2026-05-07-cross-stack-interop-ci-gating.md;
when re-added they'll reference the new path).
Plans the path from 'infra shipped' to 'full coverage with
correct behaviours'. Five new plan docs in nestsClient/plans/:
1. 2026-05-07-t16-closure-roadmap.md — index + priority order.
Three sequential steps + one independent track.
2. 2026-05-07-moq-relay-routing-investigation.md (Priority 1)
— root-cause the moq-relay 0.10.x per-broadcast subscribe-
routing race. Step-by-step: capture relay-side trace, write
minimum reproducer, file upstream OR bump moq-relay version.
Smoking gun + hypotheses already in place from the
late-join-flake investigation; this plan turns that into
actionable next steps.
3. 2026-05-07-tighten-cross-stack-assertions.md (Priority 2)
— once the routing race is closed, replace the five soft-
pass scenarios in BrowserInteropTest with hard floors.
Lists each scenario, its current soft-pass behavior, and
the proposed hard threshold.
4. 2026-05-07-cross-stack-interop-ci-gating.md (Priority 3)
— re-add the hang-interop + browser-interop GitHub Actions
jobs that were dropped in 6829ab727 / b94737de7. Includes
the exact YAML to restore and a 10/10 sweep stability bar
before merge.
5. 2026-05-07-framespergroup-production-rerun.md (independent
track) — re-run the HCgOY two-phone field tests against
current nostrnests production at multiple framesPerGroup
values to settle whether the test pin (5) and production
default (50) can converge.
Estimated total: 2.5-3.5 days of focused work to fully close
T16. After these four: I7 post-reconnect cliff and I12 GOAWAY
remain open as genuinely upstream-territory items, tracked in
their existing investigation docs.
Vitor Pamplona