Removes the `hang-interop` + `browser-interop` jobs from
`build.yml` (added in commit `21947bc5` after a 10/10 stability
sweep × 22 tests = 220/220 hard-pass). Cold-cache cost is
~10 min hang + ~13 min browser; warm-cache critical-path add
is ~6 min via the parallel browser job. Most PRs don't touch
audio / MoQ / QUIC, so paying that cost on every PR is
net-negative for the change-pattern the repo sees.
Adds `nestsClient/tests/README.md` covering:
- when to run (changes to nip53 / moq-lite session / audio
pipeline / MoqLiteNests* / ReconnectingNests* / :quic /
the sidecars themselves);
- quick-start gradle commands for hang-only, browser-only,
combined;
- prerequisites + first-run cache costs;
- all the configuration knobs (incl. the
`-DnestsHangInteropTraceRelay=true` trace-capture switch
added during the routing investigation);
- known limitations (hot-swap browser soft-pass,
framesPerGroup pin-vs-prod gap, I7 cycle 2 truncation);
- a 4-step debug recipe for triaging a flaking scenario.
Plan updates:
- `2026-05-07-t16-closure-roadmap.md` — Priority 3 marked
⏸ DEFERRED instead of ✅ CLOSED, pointing at the new README.
- `2026-05-07-cross-stack-interop-ci-gating.md` — status
changed to ⏸ DEFERRED; YAML shape preserved verbatim in the
plan for the next revisit.
- `2026-05-06-cross-stack-interop-test-results.md`,
`2026-05-06-cross-stack-interop-test-gap-matrix.md` — CI
integration § rewritten to "manual-run only" + README link.
The trace-capture instrumentation in `NativeMoqRelayHarness`
stays in place; it's useful for future flake triage even
without CI.
9.2 KiB
T16 gap matrix — wire fixes ↔ asserting interop scenarios
Status: documentation. Closes Definition of Done #5 from
2026-05-06-cross-stack-interop-test.md.
This document maps each audit-branch T# wire fix that landed in
main to ≥ 1 hang-tier and/or browser-tier interop scenario that
asserts its wire output. A regression on any T# fix would trip the
listed scenario(s) deterministically.
Source of truth for the T# series: git log --grep "fix(nests): T" against origin/main — I list only fixes that exist as concrete
commits, not the spec's aspirational T1–T14 enumeration. Scenarios
are addressed by their interop-suite ID (I1–I15) per
2026-05-06-cross-stack-interop-test.md and the
HangInteropTest / BrowserInteropTest Kotlin classes that
implement them.
Wire fixes ↔ scenarios
| T# | Fix | Commit | Asserting scenario(s) | Tier | Status |
|---|---|---|---|---|---|
| T8 | Skip BUFFER_FLAG_CODEC_CONFIG outputs in MediaCodecOpusEncoder (don't emit OpusHead as an audio frame). |
96cfa1235 |
I11 (first_audio_frame_is_not_opus_codec_config) — strips Container::Legacy and asserts the first audio frame's payload doesn't begin with the OpusHead magic. I14 (chromium_decoder_no_errors_through_warmup_window) is the browser-side mate — asserts Chromium AudioDecoder.error count is 0 across the warmup window. |
hang ✅ + browser ✅ | green |
| T10 | endGroup() on unmuted → muted transition (don't park the open uni stream when the speaker mutes). |
c23da5279 |
I3 (mid_broadcast_mute_shortens_decoded_pcm) hang-tier + chromium_listener_mid_broadcast_mute_shortens_pcm browser-tier. Asserts the listener-side decoded PCM has a sample-count deficit consistent with stream FIN, NOT embedded zeros. A regression to "push zeros instead of FIN" would trip the upper bound. |
hang ✅ + browser ✅ | green |
| T11 | Drop bestEffort = true on moq-lite group uni streams (unreliable streams behave irregularly under loss). |
7e76ab113 |
I9 (packet_loss_1pct_does_not_kill_audio) hang-tier + chromium_listener_packet_loss_1pct_does_not_kill_audio browser-tier. Drives the QUIC client through udp-loss-shim at 1 % loss; asserts the FFT peak intact. With bestEffort = true re-introduced, frames lost on dropped packets would NOT be retransmitted. |
hang ✅ + browser ✅ | green |
| T12 | Carry audio group sequence across hot-swaps (don't reset to 0 on speaker re-issuance — listener decoder caches by group ordering). | be4e0b9f9 |
I5 (speaker_hot_swap_does_not_crash) hang-tier + chromium_listener_speaker_hot_swap_does_not_crash browser-tier. Speaker calls connectReconnectingSpeaker mid-broadcast; asserts the listener sees no broadcast end and the post-swap window decodes cleanly with the 440 Hz peak intact. |
hang ✅ + browser ✅ | green |
| T13 | Reset Opus decoder on publisher boundary in NestPlayer (so the new publisher's pre-roll doesn't start mid-frame on stale decoder state). |
4714e3c72 |
I7 hang-tier (rust_hang_publish_reconnect_kotlin_listener_recovers in HangInteropReverseTest) — Rust hang-publish cycles its session at T+2.5 s. I7 reverse browser (chromium_publisher_reconnect_kotlin_listener_recovers) — Chromium publishes via publish.ts reconnect mode. Both assert ≥ 2.5 s of decoded mono PCM with the 440 Hz peak intact across the cycle. |
hang ✅ + browser ✅ | green |
| T14 | Recognise GOAWAY control type instead of silent FIN. |
73722d2ad |
N/A in moq-lite-03. moq-lite has no GOAWAY frame on the wire; the fix protects the IETF moq-transport-17 control-decoder path (MoqSession.kt:417). I12 was originally specced for this but doesn't apply — see 2026-05-06-cross-stack-interop-test-results.md's I12 section. The IETF code path is exercised by the existing MoqCodecTest unit test (unknown_control_type_skips_message_without_corruption) only — no cross-stack scenario covers it because no cross-stack peer speaks IETF moq-transport. |
unit-test only | green |
Aspirational T1–T7, T9, T15
The spec's "T1–T14" enumeration is not all wire fixes. Searching
main for fix(nests): T1 … T7, T9, T15 returns no commits;
those numbers existed in the audit's findings list but didn't
crystallise into named patches before audit closure. If a future
fix re-uses one of those numbers, this matrix should be updated to
record the asserting scenario(s) alongside the listed T# above.
The spec's claim "Catches every audio-path wire regression (T1–T14)
with at least one cross-stack scenario" should be read as: catches
every audio-path wire regression that has a concrete fix in main,
which is the T8 / T10–T14 set.
Spec scenario ↔ T# reverse index
For maintainers reading the test code who want to know which fix each scenario protects:
| Scenario | Protects |
|---|---|
| I1 (forward 440 Hz mono) | Baseline path: T8 + T11 (frames carry pristine Opus over reliable streams). A break of either trips the FFT or sample-count assertion. |
| I2 (late-join) | T11 implicitly (streams must arrive in order without RST under no-loss). |
| I3 (mute window) | T10 explicitly. |
| I4 fwd (stereo 440/660) | Stereo plumbing through AudioBroadcastConfig (PR #2755) — not a T-series fix; protects the per-channel catalog → encoder pipeline. |
| I4 rev (stereo Rust → Kotlin) | Listener stereo decode path. |
| I5 (speaker hot-swap) | T12 explicitly (hang + browser tiers). |
| I6 (multi-listener) | T11 fan-out behaviour. |
| I7 (publisher reconnect) | T13 explicitly. Hang-tier exercises Rust hang-publish reconnect; browser-tier exercises Chromium publish.ts reconnect. |
| I8 (SubscribeDrop on unknown track) | Subscribe rejection handling — moq-lite-03 protocol-level guard, not a T-series fix. |
| I9 (1 % packet loss) | T11 explicitly (hang + browser tiers). |
| I10 (60 s long broadcast) | framesPerGroup cadence interaction at scale (see 2026-05-07-framespergroup-reconciliation.md). |
| I11 (wire-byte capture) | T8 explicitly. |
| I12 (Goaway) | N/A in moq-lite-03; T14 is exercised only by the IETF moq-transport unit test path. |
I13 (browser framesPerGroup=50 long broadcast) |
framesPerGroup cadence interaction at scale on the browser path. Note: spec asked for framesPerGroup = 50; local relay's per-stream byte cliff blocks that, so the test pins 5 — see 2026-05-07-framespergroup-reconciliation.md. |
| I14 (WebCodecs warmup × CSD-skip) | T8 browser-side mate of I11. Asserts AudioDecoder.error count is 0; a OpusHead leak would fail. |
| I15 (Chromium ALPN round-trip) | moq-lite ALPN drift detection. |
Coverage state
All T-series wire fixes (T8, T10–T14) have ≥ 1 cross-stack asserting scenario landed. Both hang-tier AND browser-tier mates exist for T8/T10/T11/T12/T13. T14 (GOAWAY) only applies to the IETF moq-transport target which the production stack doesn't use.
DoD #5 (gap matrix coverage) closed.
History notes (2026-05-07):
- Five browser-tier scenarios used to soft-pass listener-side
0-frame outcomes during a flake we attributed to a moq-relay
routing race; trace capture later disproved that hypothesis
and a
:quicfix onorigin/mainclosed it (see2026-05-07-moq-relay-routing-investigation.md§ Closure). Hard floors landed in2026-05-07-tighten-cross-stack-assertions.mdafter the merge: late-join (≥ 1.5 s after warmup), mute-window (≥ 2.5 slower +< 5.5 supper kept), I14 (decoderOutputs ≥ 4), stereo (≥ 1 s × 2 ch), packet-loss (≥ 0.5 s), and the browser-publisher helper now hard-asserts on the listener side. Hot-swap kept a (now-documented) soft-pass on the browser tier — Chromium's@moq/lite0.2.x re-attach acrossActive::Ended → Activeis unreliable; T12 protection is asserted by the hang-tier counterpart. - CI gating reached green-on-the-rig (10/10 × 22 = 220/220)
in commit
21947bc5, then was deferred on cost grounds — both suites run manually now pernestsClient/tests/README.md. YAML shape preserved in2026-05-07-cross-stack-interop-ci-gating.mdfor the next revisit.
Files referenced
nestsClient/plans/2026-05-06-cross-stack-interop-test.md(spec — Definition of Done #5)nestsClient/plans/2026-05-06-cross-stack-interop-test-results.md(results — what landed)nestsClient/plans/2026-05-07-framespergroup-reconciliation.md(cadence reconciliation)nestsClient/plans/2026-05-07-i7-post-reconnect-cliff-investigation.md(I7 cycle-2 cliff)nestsClient/plans/2026-05-07-late-join-catalog-flake-investigation.md(relay routing flake)nestsClient/plans/2026-05-07-t16-closure-roadmap.md(next steps)nestsClient/src/jvmTest/kotlin/com/vitorpamplona/nestsclient/interop/native/HangInteropTest.ktnestsClient/src/jvmTest/kotlin/com/vitorpamplona/nestsclient/interop/native/HangInteropReverseTest.ktnestsClient/src/jvmTest/kotlin/com/vitorpamplona/nestsclient/interop/native/HangInteropMultiListenerTest.ktnestsClient/src/jvmTest/kotlin/com/vitorpamplona/nestsclient/interop/native/BrowserInteropTest.kt- T# fix commits:
96cfa1235(T8),c23da5279(T10),7e76ab113(T11),be4e0b9f9(T12),4714e3c72(T13),73722d2ad(T14)