Claude
15b8560547
Removes the `hang-interop` + `browser-interop` jobs from
`build.yml` (added in commit `21947bc5` after a 10/10 stability
sweep × 22 tests = 220/220 hard-pass). Cold-cache cost is
~10 min hang + ~13 min browser; warm-cache critical-path add
is ~6 min via the parallel browser job. Most PRs don't touch
audio / MoQ / QUIC, so paying that cost on every PR is
net-negative for the change-pattern the repo sees.
Adds `nestsClient/tests/README.md` covering:
- when to run (changes to nip53 / moq-lite session / audio
pipeline / MoqLiteNests* / ReconnectingNests* / :quic /
the sidecars themselves);
- quick-start gradle commands for hang-only, browser-only,
combined;
- prerequisites + first-run cache costs;
- all the configuration knobs (incl. the
`-DnestsHangInteropTraceRelay=true` trace-capture switch
added during the routing investigation);
- known limitations (hot-swap browser soft-pass,
framesPerGroup pin-vs-prod gap, I7 cycle 2 truncation);
- a 4-step debug recipe for triaging a flaking scenario.
Plan updates:
- `2026-05-07-t16-closure-roadmap.md` — Priority 3 marked
⏸ DEFERRED instead of ✅ CLOSED, pointing at the new README.
- `2026-05-07-cross-stack-interop-ci-gating.md` — status
changed to ⏸ DEFERRED; YAML shape preserved verbatim in the
plan for the next revisit.
- `2026-05-06-cross-stack-interop-test-results.md`,
`2026-05-06-cross-stack-interop-test-gap-matrix.md` — CI
integration § rewritten to "manual-run only" + README link.
The trace-capture instrumentation in `NativeMoqRelayHarness`
stays in place; it's useful for future flake triage even
without CI.
116 lines
9.2 KiB
Markdown
116 lines
9.2 KiB
Markdown
# T16 gap matrix — wire fixes ↔ asserting interop scenarios
|
||
|
||
**Status:** documentation. Closes Definition of Done #5 from
|
||
`2026-05-06-cross-stack-interop-test.md`.
|
||
|
||
This document maps each audit-branch T# wire fix that landed in
|
||
`main` to ≥ 1 hang-tier and/or browser-tier interop scenario that
|
||
asserts its wire output. A regression on any T# fix would trip the
|
||
listed scenario(s) deterministically.
|
||
|
||
**Source of truth for the T# series:** `git log --grep "fix(nests):
|
||
T"` against `origin/main` — I list only fixes that exist as concrete
|
||
commits, not the spec's aspirational T1–T14 enumeration. Scenarios
|
||
are addressed by their interop-suite ID (I1–I15) per
|
||
`2026-05-06-cross-stack-interop-test.md` and the
|
||
`HangInteropTest` / `BrowserInteropTest` Kotlin classes that
|
||
implement them.
|
||
|
||
## Wire fixes ↔ scenarios
|
||
|
||
| T# | Fix | Commit | Asserting scenario(s) | Tier | Status |
|
||
|---|---|---|---|---|---|
|
||
| **T8** | Skip `BUFFER_FLAG_CODEC_CONFIG` outputs in `MediaCodecOpusEncoder` (don't emit `OpusHead` as an audio frame). | `96cfa1235` | **I11** (`first_audio_frame_is_not_opus_codec_config`) — strips Container::Legacy and asserts the first audio frame's payload doesn't begin with the `OpusHead` magic. **I14** (`chromium_decoder_no_errors_through_warmup_window`) is the browser-side mate — asserts Chromium `AudioDecoder.error` count is 0 across the warmup window. | hang ✅ + browser ✅ | **green** |
|
||
| **T10** | `endGroup()` on unmuted → muted transition (don't park the open uni stream when the speaker mutes). | `c23da5279` | **I3** (`mid_broadcast_mute_shortens_decoded_pcm`) hang-tier + `chromium_listener_mid_broadcast_mute_shortens_pcm` browser-tier. Asserts the listener-side decoded PCM has a sample-count deficit consistent with stream FIN, NOT embedded zeros. A regression to "push zeros instead of FIN" would trip the upper bound. | hang ✅ + browser ✅ | **green** |
|
||
| **T11** | Drop `bestEffort = true` on moq-lite group uni streams (unreliable streams behave irregularly under loss). | `7e76ab113` | **I9** (`packet_loss_1pct_does_not_kill_audio`) hang-tier + `chromium_listener_packet_loss_1pct_does_not_kill_audio` browser-tier. Drives the QUIC client through `udp-loss-shim` at 1 % loss; asserts the FFT peak intact. With `bestEffort = true` re-introduced, frames lost on dropped packets would NOT be retransmitted. | hang ✅ + browser ✅ | **green** |
|
||
| **T12** | Carry audio group sequence across hot-swaps (don't reset to 0 on speaker re-issuance — listener decoder caches by group ordering). | `be4e0b9f9` | **I5** (`speaker_hot_swap_does_not_crash`) hang-tier + `chromium_listener_speaker_hot_swap_does_not_crash` browser-tier. Speaker calls `connectReconnectingSpeaker` mid-broadcast; asserts the listener sees no broadcast end and the post-swap window decodes cleanly with the 440 Hz peak intact. | hang ✅ + browser ✅ | **green** |
|
||
| **T13** | Reset Opus decoder on publisher boundary in `NestPlayer` (so the new publisher's pre-roll doesn't start mid-frame on stale decoder state). | `4714e3c72` | **I7** hang-tier (`rust_hang_publish_reconnect_kotlin_listener_recovers` in `HangInteropReverseTest`) — Rust hang-publish cycles its session at T+2.5 s. **I7 reverse browser** (`chromium_publisher_reconnect_kotlin_listener_recovers`) — Chromium publishes via `publish.ts` reconnect mode. Both assert ≥ 2.5 s of decoded mono PCM with the 440 Hz peak intact across the cycle. | hang ✅ + browser ✅ | **green** |
|
||
| **T14** | Recognise `GOAWAY` control type instead of silent FIN. | `73722d2ad` | **N/A in moq-lite-03.** moq-lite has no `GOAWAY` frame on the wire; the fix protects the IETF moq-transport-17 control-decoder path (`MoqSession.kt:417`). I12 was originally specced for this but doesn't apply — see `2026-05-06-cross-stack-interop-test-results.md`'s I12 section. The IETF code path is exercised by the existing `MoqCodecTest` unit test (`unknown_control_type_skips_message_without_corruption`) only — no cross-stack scenario covers it because no cross-stack peer speaks IETF moq-transport. | unit-test only | **green** |
|
||
|
||
## Aspirational T1–T7, T9, T15
|
||
|
||
The spec's "T1–T14" enumeration is not all wire fixes. Searching
|
||
`main` for `fix(nests): T1` … `T7`, `T9`, `T15` returns no commits;
|
||
those numbers existed in the audit's findings list but didn't
|
||
crystallise into named patches before audit closure. If a future
|
||
fix re-uses one of those numbers, this matrix should be updated to
|
||
record the asserting scenario(s) alongside the listed T# above.
|
||
|
||
The spec's claim "Catches every audio-path wire regression (T1–T14)
|
||
with at least one cross-stack scenario" should be read as: catches
|
||
every audio-path wire regression that has a concrete fix in `main`,
|
||
which is the T8 / T10–T14 set.
|
||
|
||
## Spec scenario ↔ T# reverse index
|
||
|
||
For maintainers reading the test code who want to know *which* fix
|
||
each scenario protects:
|
||
|
||
| Scenario | Protects |
|
||
|---|---|
|
||
| **I1** (forward 440 Hz mono) | Baseline path: T8 + T11 (frames carry pristine Opus over reliable streams). A break of either trips the FFT or sample-count assertion. |
|
||
| **I2** (late-join) | T11 implicitly (streams must arrive in order without RST under no-loss). |
|
||
| **I3** (mute window) | **T10** explicitly. |
|
||
| **I4 fwd** (stereo 440/660) | Stereo plumbing through `AudioBroadcastConfig` (PR #2755) — not a T-series fix; protects the per-channel catalog → encoder pipeline. |
|
||
| **I4 rev** (stereo Rust → Kotlin) | Listener stereo decode path. |
|
||
| **I5** (speaker hot-swap) | **T12** explicitly (hang + browser tiers). |
|
||
| **I6** (multi-listener) | T11 fan-out behaviour. |
|
||
| **I7** (publisher reconnect) | **T13** explicitly. Hang-tier exercises Rust hang-publish reconnect; browser-tier exercises Chromium publish.ts reconnect. |
|
||
| **I8** (SubscribeDrop on unknown track) | Subscribe rejection handling — moq-lite-03 protocol-level guard, not a T-series fix. |
|
||
| **I9** (1 % packet loss) | **T11** explicitly (hang + browser tiers). |
|
||
| **I10** (60 s long broadcast) | `framesPerGroup` cadence interaction at scale (see `2026-05-07-framespergroup-reconciliation.md`). |
|
||
| **I11** (wire-byte capture) | **T8** explicitly. |
|
||
| **I12** (Goaway) | N/A in moq-lite-03; **T14** is exercised only by the IETF moq-transport unit test path. |
|
||
| **I13** (browser `framesPerGroup=50` long broadcast) | `framesPerGroup` cadence interaction at scale on the browser path. Note: spec asked for `framesPerGroup = 50`; local relay's per-stream byte cliff blocks that, so the test pins `5` — see `2026-05-07-framespergroup-reconciliation.md`. |
|
||
| **I14** (WebCodecs warmup × CSD-skip) | **T8** browser-side mate of I11. Asserts `AudioDecoder.error` count is 0; a `OpusHead` leak would fail. |
|
||
| **I15** (Chromium ALPN round-trip) | moq-lite ALPN drift detection. |
|
||
|
||
## Coverage state
|
||
|
||
All T-series wire fixes (T8, T10–T14) have ≥ 1 cross-stack
|
||
asserting scenario landed. Both hang-tier AND browser-tier
|
||
mates exist for T8/T10/T11/T12/T13. T14 (GOAWAY) only applies
|
||
to the IETF moq-transport target which the production stack
|
||
doesn't use.
|
||
|
||
DoD #5 (gap matrix coverage) closed.
|
||
|
||
**History notes (2026-05-07):**
|
||
|
||
- Five browser-tier scenarios used to soft-pass listener-side
|
||
0-frame outcomes during a flake we attributed to a moq-relay
|
||
routing race; trace capture later disproved that hypothesis
|
||
and a `:quic` fix on `origin/main` closed it (see
|
||
`2026-05-07-moq-relay-routing-investigation.md` § Closure).
|
||
Hard floors landed in
|
||
`2026-05-07-tighten-cross-stack-assertions.md` after the merge:
|
||
late-join (`≥ 1.5 s after warmup`), mute-window (`≥ 2.5 s`
|
||
lower + `< 5.5 s` upper kept), I14 (`decoderOutputs ≥ 4`),
|
||
stereo (`≥ 1 s × 2 ch`), packet-loss (`≥ 0.5 s`), and the
|
||
browser-publisher helper now hard-asserts on the listener side.
|
||
Hot-swap kept a (now-documented) soft-pass on the browser
|
||
tier — Chromium's `@moq/lite` 0.2.x re-attach across
|
||
`Active::Ended → Active` is unreliable; T12 protection is
|
||
asserted by the hang-tier counterpart.
|
||
- CI gating reached green-on-the-rig (10/10 × 22 = 220/220)
|
||
in commit `21947bc5`, then was deferred on cost grounds —
|
||
both suites run manually now per
|
||
[`nestsClient/tests/README.md`](../tests/README.md). YAML
|
||
shape preserved in
|
||
`2026-05-07-cross-stack-interop-ci-gating.md` for the next
|
||
revisit.
|
||
|
||
## Files referenced
|
||
|
||
- `nestsClient/plans/2026-05-06-cross-stack-interop-test.md` (spec — Definition of Done #5)
|
||
- `nestsClient/plans/2026-05-06-cross-stack-interop-test-results.md` (results — what landed)
|
||
- `nestsClient/plans/2026-05-07-framespergroup-reconciliation.md` (cadence reconciliation)
|
||
- `nestsClient/plans/2026-05-07-i7-post-reconnect-cliff-investigation.md` (I7 cycle-2 cliff)
|
||
- `nestsClient/plans/2026-05-07-late-join-catalog-flake-investigation.md` (relay routing flake)
|
||
- `nestsClient/plans/2026-05-07-t16-closure-roadmap.md` (next steps)
|
||
- `nestsClient/src/jvmTest/kotlin/com/vitorpamplona/nestsclient/interop/native/HangInteropTest.kt`
|
||
- `nestsClient/src/jvmTest/kotlin/com/vitorpamplona/nestsclient/interop/native/HangInteropReverseTest.kt`
|
||
- `nestsClient/src/jvmTest/kotlin/com/vitorpamplona/nestsclient/interop/native/HangInteropMultiListenerTest.kt`
|
||
- `nestsClient/src/jvmTest/kotlin/com/vitorpamplona/nestsclient/interop/native/BrowserInteropTest.kt`
|
||
- T# fix commits: `96cfa1235` (T8), `c23da5279` (T10), `7e76ab113` (T11), `be4e0b9f9` (T12), `4714e3c72` (T13), `73722d2ad` (T14)
|