Removes the `hang-interop` + `browser-interop` jobs from
`build.yml` (added in commit `21947bc5` after a 10/10 stability
sweep × 22 tests = 220/220 hard-pass). Cold-cache cost is
~10 min hang + ~13 min browser; warm-cache critical-path add
is ~6 min via the parallel browser job. Most PRs don't touch
audio / MoQ / QUIC, so paying that cost on every PR is
net-negative for the change-pattern the repo sees.
Adds `nestsClient/tests/README.md` covering:
- when to run (changes to nip53 / moq-lite session / audio
pipeline / MoqLiteNests* / ReconnectingNests* / :quic /
the sidecars themselves);
- quick-start gradle commands for hang-only, browser-only,
combined;
- prerequisites + first-run cache costs;
- all the configuration knobs (incl. the
`-DnestsHangInteropTraceRelay=true` trace-capture switch
added during the routing investigation);
- known limitations (hot-swap browser soft-pass,
framesPerGroup pin-vs-prod gap, I7 cycle 2 truncation);
- a 4-step debug recipe for triaging a flaking scenario.
Plan updates:
- `2026-05-07-t16-closure-roadmap.md` — Priority 3 marked
⏸ DEFERRED instead of ✅ CLOSED, pointing at the new README.
- `2026-05-07-cross-stack-interop-ci-gating.md` — status
changed to ⏸ DEFERRED; YAML shape preserved verbatim in the
plan for the next revisit.
- `2026-05-06-cross-stack-interop-test-results.md`,
`2026-05-06-cross-stack-interop-test-gap-matrix.md` — CI
integration § rewritten to "manual-run only" + README link.
The trace-capture instrumentation in `NativeMoqRelayHarness`
stays in place; it's useful for future flake triage even
without CI.
6.0 KiB
T16 closure roadmap — full coverage with correct behaviours
Goal state. Every spec'd cross-stack scenario green in suite-mode sweeps, asserting its full design intent (no soft-passes, no vacuous threshold loosening), with CI gating live and stable.
Where we are. The merged claude/cross-stack-interop-test-XAbYB
branch ships 22 of 23 spec'd scenarios; each passes individually.
Suite-mode runs hit a residual moq-relay 0.10.x routing race on a
specific subset (~40-60% flake rate). Five scenarios soft-pass the
listener side as a known-flake mitigation. CI is intentionally
unwired pending stability.
This roadmap takes the suite from "passes individually" to "passes in suite + CI" through three sequential plans. None should be parallelized — each unblocks the next.
Priority 1 — 2026-05-07-moq-relay-routing-investigation.md ✅ CLOSED
Closed 2026-05-07 by merging
origin/main(five:quiccommits:2a4c07ae,d5c854be,b622d0c9,86a4727e,31d19258). The flake was a:quicpacket-acceptance bug, not a moq-relay routing race. Step 1 trace capture in this branch first disproved the moq-relay-routing hypothesis (relay correctly forwards the upstream SUBSCRIBE); the QUIC team's separate work landed in main between the merge base and pickup and incidentally closed the speaker-side post-handshake bidi drop.
What landed.
- A 5-commit merge from
origin/maincarrying ALPN-list threading, PTO STREAM retransmits, RFC 9001 §6 1-RTT key update, multiconnect/multiplex pacing, and qlog flush. - Per-test moq-relay TRACE capture instrumentation
(commit
d7f87971) — kept in place; useful for follow-up regression triage. - Cross-stack trace artefacts preserved at
nestsClient/plans/artefacts/2026-05-07-routing-race-disproven/for post-mortem reference.
Acceptance bar met. 5/5 sweep BUILD SUCCESSFUL on HangInteropTest with their current soft-pass assertions intact. 55/55 tests pass.
Priority 2 — 2026-05-07-tighten-cross-stack-assertions.md ✅ CLOSED
Closed 2026-05-07 by commits
04be38ad(initial tightening) and029329af(recalibration of two floors against empirical post-merge steady state). Verification sweep: 5/5 BUILD SUCCESSFUL × 22 tests = 110/110 hard-pass on./gradlew :nestsClient:jvmTest --tests HangInteropTest --tests BrowserInteropTest -DnestsHangInterop=true -DnestsBrowserInterop=true --rerun-tasks.
What landed.
- 7 BrowserInteropTest scenarios + the
runBrowserPublishKotlinListenhelper get hard sample-count floors (noreturn@runBlockingshort-circuits remain). - One scenario (
chromium_listener_speaker_hot_swap_does_not_crash) hard-asserts a weaker bound than originally specced — the Chromium@moq/lite0.2.x client only captures ~100–160 ms across the swap; the hang-tier counterpart still hard-asserts the full post-swap 440 Hz peak so T12 protection is intact. Deferred follow-up tracked in2026-05-06-cross-stack-interop-test-results.md. - Gap matrix updated.
Acceptance bar met. 5/5 sweep with hard assertions.
Priority 3 — 2026-05-07-cross-stack-interop-ci-gating.md ⏸ DEFERRED (manual run only)
Briefly wired then reverted, 2026-05-07. Commit
21947bc5re-added both jobs tobuild.yml; verification gave 10/10 sweep × 22 tests = 220/220 hard-pass on the agent rig. A subsequent maintainer review judged the cold-cache cost (~10 min hang, ~13 min browser) too high for the change-pattern (most PRs don't touch audio / MoQ / QUIC) and removed the jobs.The suites stay green locally and are intended to run before merging any change that touches the audio / moq-lite /
:quicstack. Documentation:nestsClient/tests/README.mdcovers when/how/prerequisites/debugging.The CI-gating plan is kept around in case the wallclock cost calculus changes (e.g., a much faster runner, or a smaller opt-in subset of the suite that runs in <2 min).
Independent track — 2026-05-07-framespergroup-production-rerun.md
This one doesn't block the closure roadmap. It can run any time after Priority 1 is done; it settles whether the test pin (5) and production default (50) can converge, or whether they must remain different. Either outcome is shippable.
What lands.
- Logcat data from a fresh two-phone field test against current
nostrnests production at multiple
framesPerGroupvalues. - A data-driven decision on whether to change the production default, the test pin, or neither.
Why it's parallelizable. Doesn't gate the test suite or CI;
it gates a one-line code change to NestMoqLiteBroadcaster's
default constant.
After all four close — what remains
Two open items, both genuinely upstream:
-
I7 post-reconnect listener cliff —
2026-05-07-i7-post-reconnect-cliff-investigation.md. The I7 reverse scenario passes its 2.5 s threshold but a regression test of "all post-reconnect data arrives" would require the moq-relay 0.10.x per-broadcast forward queue fix. Same upstream class as the routing race. -
I12 GOAWAY — only re-emerges if an IETF moq-transport target lands (currently moq-lite-03 only). Tracked in
2026-05-06-cross-stack-interop-test-results.md.
Beyond those: T16 reaches "full coverage with correct behaviours" when this roadmap closes.
Estimated wallclock
- Priority 1: 1–2 days (depends on whether upstream version bump fixes it, or we have to file + wait for upstream).
- Priority 2: 0.5 day (mechanical replacement of soft-passes with floors, plus rerun verification).
- Priority 3: 0.5 day (re-add CI jobs, run the 10× sweep, merge).
- Independent track (framesPerGroup): 0.5 day (needs prod-rig access).
Total: 2.5–3.5 days of focused work to take T16 from "infra shipped" to "fully closed".
Plan files
2026-05-07-moq-relay-routing-investigation.md2026-05-07-tighten-cross-stack-assertions.md2026-05-07-cross-stack-interop-ci-gating.md2026-05-07-framespergroup-production-rerun.md- (this file)
2026-05-07-t16-closure-roadmap.md