ci(nests): defer cross-stack interop CI; document manual run
Removes the `hang-interop` + `browser-interop` jobs from
`build.yml` (added in commit `21947bc5` after a 10/10 stability
sweep × 22 tests = 220/220 hard-pass). Cold-cache cost is
~10 min hang + ~13 min browser; warm-cache critical-path add
is ~6 min via the parallel browser job. Most PRs don't touch
audio / MoQ / QUIC, so paying that cost on every PR is
net-negative for the change-pattern the repo sees.
Adds `nestsClient/tests/README.md` covering:
- when to run (changes to nip53 / moq-lite session / audio
pipeline / MoqLiteNests* / ReconnectingNests* / :quic /
the sidecars themselves);
- quick-start gradle commands for hang-only, browser-only,
combined;
- prerequisites + first-run cache costs;
- all the configuration knobs (incl. the
`-DnestsHangInteropTraceRelay=true` trace-capture switch
added during the routing investigation);
- known limitations (hot-swap browser soft-pass,
framesPerGroup pin-vs-prod gap, I7 cycle 2 truncation);
- a 4-step debug recipe for triaging a flaking scenario.
Plan updates:
- `2026-05-07-t16-closure-roadmap.md` — Priority 3 marked
⏸ DEFERRED instead of ✅ CLOSED, pointing at the new README.
- `2026-05-07-cross-stack-interop-ci-gating.md` — status
changed to ⏸ DEFERRED; YAML shape preserved verbatim in the
plan for the next revisit.
- `2026-05-06-cross-stack-interop-test-results.md`,
`2026-05-06-cross-stack-interop-test-gap-matrix.md` — CI
integration § rewritten to "manual-run only" + README link.
The trace-capture instrumentation in `NativeMoqRelayHarness`
stays in place; it's useful for future flake triage even
without CI.
This commit is contained in:
@@ -92,9 +92,13 @@ DoD #5 (gap matrix coverage) closed.
|
||||
tier — Chromium's `@moq/lite` 0.2.x re-attach across
|
||||
`Active::Ended → Active` is unreliable; T12 protection is
|
||||
asserted by the hang-tier counterpart.
|
||||
- CI gating wired in
|
||||
`2026-05-07-cross-stack-interop-ci-gating.md` after a 10/10
|
||||
sweep × 22 tests = 220/220 pass stability bar.
|
||||
- CI gating reached green-on-the-rig (10/10 × 22 = 220/220)
|
||||
in commit `21947bc5`, then was deferred on cost grounds —
|
||||
both suites run manually now per
|
||||
[`nestsClient/tests/README.md`](../tests/README.md). YAML
|
||||
shape preserved in
|
||||
`2026-05-07-cross-stack-interop-ci-gating.md` for the next
|
||||
revisit.
|
||||
|
||||
## Files referenced
|
||||
|
||||
|
||||
@@ -440,25 +440,30 @@ relay-side timing under load.
|
||||
|
||||
## CI integration
|
||||
|
||||
**Wired** as of 2026-05-07 (commit `21947bc5`). Both
|
||||
`hang-interop` and `browser-interop` jobs in
|
||||
`.github/workflows/build.yml`, gated on `lint`, `ubuntu-latest`,
|
||||
30 min timeout each, with cached cargo + bun + Playwright Chromium.
|
||||
**Manual-run only** (deferred from CI on cost grounds). Both
|
||||
suites are kept green and locally invokable; developer-facing
|
||||
docs at [`nestsClient/tests/README.md`](../tests/README.md)
|
||||
cover when/how/prerequisites.
|
||||
|
||||
Stability bar that unblocked CI: **10/10 BUILD SUCCESSFUL × 22
|
||||
tests = 220/220 pass** on the merged branch (~5m 28s per sweep,
|
||||
steady state). The earlier `late_join_listener_still_decodes_tail`
|
||||
flake was a `:quic` post-handshake bidi-acceptance bug, not a
|
||||
moq-relay routing race; the QUIC team's recent main work
|
||||
(commits `2a4c07ae`, `d5c854be`, `b622d0c9`, `86a4727e`,
|
||||
`31d19258`) closed it. See
|
||||
`2026-05-07-moq-relay-routing-investigation.md` § Closure for
|
||||
the trace evidence and
|
||||
`2026-05-07-t16-closure-roadmap.md` for the rolled-up status.
|
||||
Brief history:
|
||||
|
||||
The 2-week post-merge CI green-rate watch is on the maintainer
|
||||
who lands this: ≥ 95 % required per the plan's stability gate;
|
||||
otherwise pull both jobs and reopen the routing investigation.
|
||||
- 2026-05-07: 10/10 sweep × 22 tests = 220/220 hard-pass
|
||||
established a working stability bar after the `:quic`
|
||||
post-handshake bidi-drop fix landed via `origin/main` (commits
|
||||
`2a4c07ae`, `d5c854be`, `b622d0c9`, `86a4727e`, `31d19258`).
|
||||
- Commit `21947bc5` re-added the `hang-interop` +
|
||||
`browser-interop` jobs to `.github/workflows/build.yml`.
|
||||
- Maintainer review then deferred CI gating on cost grounds
|
||||
(cold cache ~10 min hang, ~13 min browser; most PRs don't
|
||||
touch audio / MoQ / QUIC). Both jobs were removed; the YAML
|
||||
shape stays preserved in the closed-but-deferred plan
|
||||
[`2026-05-07-cross-stack-interop-ci-gating.md`](2026-05-07-cross-stack-interop-ci-gating.md)
|
||||
for re-evaluation if the cost calculus changes.
|
||||
|
||||
The trace-capture instrumentation
|
||||
(`-DnestsHangInteropTraceRelay=true`) added during the routing
|
||||
investigation stays in place — useful when triaging a future
|
||||
flake.
|
||||
|
||||
## Pending follow-ups
|
||||
|
||||
|
||||
@@ -1,19 +1,26 @@
|
||||
# Plan: wire CI gating for the cross-stack interop suite
|
||||
|
||||
**Status:** ✅ CLOSED 2026-05-07. Both jobs wired in commit
|
||||
`21947bc5` (path-tweaked from the original removed shape per
|
||||
the `nestsClient/tests/browser-interop/` move). Stability-bar
|
||||
sweep: **10/10 BUILD SUCCESSFUL × 22 tests = 220/220 pass.**
|
||||
Acceptance bar from the roadmap met.
|
||||
**Status:** ⏸ DEFERRED 2026-05-07.
|
||||
The infrastructure to wire CI is ready — both jobs landed in
|
||||
commit `21947bc5` and a 10/10 stability sweep × 22 tests =
|
||||
220/220 passed on the agent rig. A maintainer review then
|
||||
judged the wallclock cost too high for the change-pattern and
|
||||
removed the jobs (cold cache: ~10 min hang, ~13 min browser;
|
||||
most PRs don't touch audio / MoQ / QUIC).
|
||||
|
||||
The suites are now run manually before merging changes that
|
||||
touch the relevant code paths. Developer-facing docs:
|
||||
[`../tests/README.md`](../tests/README.md). This plan stays
|
||||
in the tree for the next time someone wants to revisit CI
|
||||
gating — the YAML shapes below are the exact reverse-merge
|
||||
target, and the stability bar (10/10 sweep) has already been
|
||||
demonstrated on this branch.
|
||||
|
||||
**Depended on:**
|
||||
- `2026-05-07-moq-relay-routing-investigation.md` closed (✅)
|
||||
- `2026-05-07-tighten-cross-stack-assertions.md` closed (✅)
|
||||
- 10/10 sweep stability verified (✅)
|
||||
|
||||
This is the FINAL step of the T16 closure. With stable hard-pass
|
||||
suites, CI gating becomes safe and meaningful.
|
||||
|
||||
## What's needed
|
||||
|
||||
### A) `.github/workflows/build.yml` — the hang-interop job
|
||||
|
||||
@@ -67,22 +67,23 @@ HangInteropTest with their current soft-pass assertions intact.
|
||||
|
||||
**Acceptance bar met.** 5/5 sweep with hard assertions.
|
||||
|
||||
## Priority 3 — `2026-05-07-cross-stack-interop-ci-gating.md` ✅ CLOSED
|
||||
## Priority 3 — `2026-05-07-cross-stack-interop-ci-gating.md` ⏸ DEFERRED (manual run only)
|
||||
|
||||
> **Closed 2026-05-07** by commit `21947bc5`. Both
|
||||
> `hang-interop` and `browser-interop` jobs landed in
|
||||
> `.github/workflows/build.yml`, gated on `lint`,
|
||||
> `ubuntu-latest`, 30 min timeout, with cached cargo + bun +
|
||||
> Playwright Chromium. Path-tweaked from the original removed
|
||||
> shape because the browser harness moved from
|
||||
> `nestsClient-browser-interop/` to
|
||||
> `nestsClient/tests/browser-interop/` (commit `bd7b166f`).
|
||||
|
||||
**Acceptance bar met.** 10/10 sweep BUILD SUCCESSFUL × 22 tests
|
||||
= **220/220 pass** (~5m 28s steady state per sweep on the agent
|
||||
rig). The first 2 weeks of post-merge CI runs still need a
|
||||
maintainer to monitor flake rate (per the plan's "≥ 95% green
|
||||
rate" gate); pull the jobs again if it falls below.
|
||||
> **Briefly wired then reverted, 2026-05-07.** Commit `21947bc5`
|
||||
> re-added both jobs to `build.yml`; verification gave 10/10
|
||||
> sweep × 22 tests = 220/220 hard-pass on the agent rig. A
|
||||
> subsequent maintainer review judged the cold-cache cost
|
||||
> (~10 min hang, ~13 min browser) too high for the change-pattern
|
||||
> (most PRs don't touch audio / MoQ / QUIC) and removed the jobs.
|
||||
>
|
||||
> The suites stay green locally and are intended to run before
|
||||
> merging any change that touches the audio / moq-lite / `:quic`
|
||||
> stack. Documentation: [`nestsClient/tests/README.md`](../tests/README.md)
|
||||
> covers when/how/prerequisites/debugging.
|
||||
>
|
||||
> The CI-gating plan is kept around in case the wallclock cost
|
||||
> calculus changes (e.g., a much faster runner, or a smaller
|
||||
> opt-in subset of the suite that runs in <2 min).
|
||||
|
||||
## Independent track — `2026-05-07-framespergroup-production-rerun.md`
|
||||
|
||||
|
||||
Reference in New Issue
Block a user