ci(nests): defer cross-stack interop CI; document manual run
Removes the `hang-interop` + `browser-interop` jobs from
`build.yml` (added in commit `21947bc5` after a 10/10 stability
sweep × 22 tests = 220/220 hard-pass). Cold-cache cost is
~10 min hang + ~13 min browser; warm-cache critical-path add
is ~6 min via the parallel browser job. Most PRs don't touch
audio / MoQ / QUIC, so paying that cost on every PR is
net-negative for the change-pattern the repo sees.
Adds `nestsClient/tests/README.md` covering:
- when to run (changes to nip53 / moq-lite session / audio
pipeline / MoqLiteNests* / ReconnectingNests* / :quic /
the sidecars themselves);
- quick-start gradle commands for hang-only, browser-only,
combined;
- prerequisites + first-run cache costs;
- all the configuration knobs (incl. the
`-DnestsHangInteropTraceRelay=true` trace-capture switch
added during the routing investigation);
- known limitations (hot-swap browser soft-pass,
framesPerGroup pin-vs-prod gap, I7 cycle 2 truncation);
- a 4-step debug recipe for triaging a flaking scenario.
Plan updates:
- `2026-05-07-t16-closure-roadmap.md` — Priority 3 marked
⏸ DEFERRED instead of ✅ CLOSED, pointing at the new README.
- `2026-05-07-cross-stack-interop-ci-gating.md` — status
changed to ⏸ DEFERRED; YAML shape preserved verbatim in the
plan for the next revisit.
- `2026-05-06-cross-stack-interop-test-results.md`,
`2026-05-06-cross-stack-interop-test-gap-matrix.md` — CI
integration § rewritten to "manual-run only" + README link.
The trace-capture instrumentation in `NativeMoqRelayHarness`
stays in place; it's useful for future flake triage even
without CI.
This commit is contained in:
@@ -174,164 +174,6 @@ jobs:
|
|||||||
name: ${{ matrix.desktop-artifact-name }}
|
name: ${{ matrix.desktop-artifact-name }}
|
||||||
path: ${{ matrix.desktop-artifact-path }}
|
path: ${{ matrix.desktop-artifact-path }}
|
||||||
|
|
||||||
# Cross-stack interop tests (T16). Builds the Rust hang-listen +
|
|
||||||
# hang-publish sidecars (nestsClient/tests/hang-interop/), `cargo install`s
|
|
||||||
# moq-relay + moq-token-cli at the version pinned in
|
|
||||||
# `nestsClient/tests/hang-interop/REV`, then runs `:nestsClient:jvmTest
|
|
||||||
# -DnestsHangInterop=true`. See
|
|
||||||
# `nestsClient/plans/2026-05-06-cross-stack-interop-test.md` and the
|
|
||||||
# closure roadmap at
|
|
||||||
# `nestsClient/plans/2026-05-07-t16-closure-roadmap.md`.
|
|
||||||
#
|
|
||||||
# Linux-only: the cargo install of `moq-relay` 0.10.x has nontrivial
|
|
||||||
# native deps (aws-lc-sys, ring) that take 5+ min cold; we cache
|
|
||||||
# both ~/.cargo and the nestsClient/tests/hang-interop/target tree so the warm
|
|
||||||
# path is a few seconds. macOS / Windows runs would double the
|
|
||||||
# matrix cost without catching anything Linux doesn't catch — the
|
|
||||||
# protocol logic is platform-agnostic and the JNA libopus natives
|
|
||||||
# are already exercised by the unit-level JvmOpusRoundTripTest on
|
|
||||||
# the Android job.
|
|
||||||
hang-interop:
|
|
||||||
needs: lint
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
timeout-minutes: 30
|
|
||||||
steps:
|
|
||||||
- name: Checkout code
|
|
||||||
uses: actions/checkout@v6
|
|
||||||
|
|
||||||
- name: Set up JDK 21
|
|
||||||
uses: actions/setup-java@v5
|
|
||||||
with:
|
|
||||||
distribution: 'zulu'
|
|
||||||
java-version: 21
|
|
||||||
|
|
||||||
- name: Set up Gradle
|
|
||||||
uses: gradle/actions/setup-gradle@v4
|
|
||||||
with:
|
|
||||||
cache-read-only: ${{ github.ref != 'refs/heads/main' }}
|
|
||||||
|
|
||||||
# Pin Rust to ≥ 1.95 — moq-relay 0.10.25's transitive dep
|
|
||||||
# `constant_time_eq 0.4.3` requires it, and ubuntu-latest's
|
|
||||||
# default Rust version drifts.
|
|
||||||
- name: Set up Rust
|
|
||||||
uses: dtolnay/rust-toolchain@stable
|
|
||||||
|
|
||||||
# Cache cargo registry + the sidecar workspace's target/.
|
|
||||||
# First cold run takes ~6 min for the moq-relay install;
|
|
||||||
# cached runs ~30 s.
|
|
||||||
- name: Cache cargo
|
|
||||||
uses: actions/cache@v4
|
|
||||||
with:
|
|
||||||
path: |
|
|
||||||
~/.cargo/registry
|
|
||||||
~/.cargo/git
|
|
||||||
nestsClient/tests/hang-interop/target
|
|
||||||
~/.cache/amethyst-nests-interop/hang-interop-cargo
|
|
||||||
key: ${{ runner.os }}-cargo-${{ hashFiles('nestsClient/tests/hang-interop/Cargo.lock', 'nestsClient/tests/hang-interop/REV') }}
|
|
||||||
restore-keys: |
|
|
||||||
${{ runner.os }}-cargo-
|
|
||||||
|
|
||||||
- name: Run cross-stack interop suite
|
|
||||||
run: ./gradlew :nestsClient:jvmTest --tests "com.vitorpamplona.nestsclient.interop.native.HangInteropTest" -DnestsHangInterop=true
|
|
||||||
|
|
||||||
- name: Upload interop test report
|
|
||||||
uses: actions/upload-artifact@v7
|
|
||||||
if: failure()
|
|
||||||
with:
|
|
||||||
name: Hang Interop Test Reports
|
|
||||||
path: nestsClient/build/reports/tests/jvmTest/
|
|
||||||
|
|
||||||
# Phase 4 of T16: browser-side cross-stack interop. Drives a headless
|
|
||||||
# Chromium (via Playwright) running @moq/lite + @moq/hang against
|
|
||||||
# the same moq-relay subprocess the hang-interop job uses. Linux-only:
|
|
||||||
# Chromium QUIC behaviour is consistent across platforms in the
|
|
||||||
# scenarios we care about, and macOS/Windows would double the matrix
|
|
||||||
# cost without catching new defects.
|
|
||||||
#
|
|
||||||
# Inherits the cargo cache from `hang-interop` because the browser
|
|
||||||
# path still needs `moq-relay` + `hang-listen` built (the harness
|
|
||||||
# boots the same Rust subprocess). Adds bun + node_modules + Playwright
|
|
||||||
# browser caches. See:
|
|
||||||
# nestsClient/plans/2026-05-06-phase4-browser-harness.md
|
|
||||||
browser-interop:
|
|
||||||
needs: lint
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
timeout-minutes: 30
|
|
||||||
steps:
|
|
||||||
- name: Checkout code
|
|
||||||
uses: actions/checkout@v6
|
|
||||||
|
|
||||||
- name: Set up JDK 21
|
|
||||||
uses: actions/setup-java@v5
|
|
||||||
with:
|
|
||||||
distribution: 'zulu'
|
|
||||||
java-version: 21
|
|
||||||
|
|
||||||
- name: Set up Gradle
|
|
||||||
uses: gradle/actions/setup-gradle@v4
|
|
||||||
with:
|
|
||||||
cache-read-only: ${{ github.ref != 'refs/heads/main' }}
|
|
||||||
|
|
||||||
- name: Set up Rust
|
|
||||||
uses: dtolnay/rust-toolchain@stable
|
|
||||||
|
|
||||||
- name: Cache cargo
|
|
||||||
uses: actions/cache@v4
|
|
||||||
with:
|
|
||||||
path: |
|
|
||||||
~/.cargo/registry
|
|
||||||
~/.cargo/git
|
|
||||||
nestsClient/tests/hang-interop/target
|
|
||||||
~/.cache/amethyst-nests-interop/hang-interop-cargo
|
|
||||||
key: ${{ runner.os }}-cargo-${{ hashFiles('nestsClient/tests/hang-interop/Cargo.lock', 'nestsClient/tests/hang-interop/REV') }}
|
|
||||||
restore-keys: |
|
|
||||||
${{ runner.os }}-cargo-
|
|
||||||
|
|
||||||
# bun is a separate install — Playwright + the bun harness build
|
|
||||||
# both go through it. Pin the version that matches the local agent
|
|
||||||
# tooling so a CI/local skew can't surface a wire-format change
|
|
||||||
# in `bun build` output.
|
|
||||||
- name: Set up bun
|
|
||||||
uses: oven-sh/setup-bun@v2
|
|
||||||
with:
|
|
||||||
bun-version: 1.3.11
|
|
||||||
|
|
||||||
# Cache bun-installed node_modules for the browser harness.
|
|
||||||
# Keyed on package.json + bun.lock so a dep bump invalidates.
|
|
||||||
- name: Cache node_modules
|
|
||||||
uses: actions/cache@v4
|
|
||||||
with:
|
|
||||||
path: |
|
|
||||||
nestsClient/tests/browser-interop/node_modules
|
|
||||||
nestsClient/tests/browser-interop/dist
|
|
||||||
key: ${{ runner.os }}-bun-${{ hashFiles('nestsClient/tests/browser-interop/package.json', 'nestsClient/tests/browser-interop/bun.lock') }}
|
|
||||||
|
|
||||||
# Cache Playwright's browser cache (~/.cache/ms-playwright/chromium-*).
|
|
||||||
# The cold install of Chromium is ~200 MB and 30–60 s; cached runs
|
|
||||||
# are essentially instant.
|
|
||||||
- name: Cache Playwright browsers
|
|
||||||
uses: actions/cache@v4
|
|
||||||
with:
|
|
||||||
path: ~/.cache/ms-playwright
|
|
||||||
key: ${{ runner.os }}-playwright-${{ hashFiles('nestsClient/tests/browser-interop/package.json') }}
|
|
||||||
|
|
||||||
- name: Run browser cross-stack interop suite
|
|
||||||
run: |
|
|
||||||
./gradlew :nestsClient:jvmTest \
|
|
||||||
--tests "com.vitorpamplona.nestsclient.interop.native.BrowserInteropTest" \
|
|
||||||
-DnestsHangInterop=true \
|
|
||||||
-DnestsBrowserInterop=true
|
|
||||||
|
|
||||||
- name: Upload browser interop test report
|
|
||||||
uses: actions/upload-artifact@v7
|
|
||||||
if: failure()
|
|
||||||
with:
|
|
||||||
name: Browser Interop Test Reports
|
|
||||||
path: |
|
|
||||||
nestsClient/build/reports/tests/jvmTest/
|
|
||||||
nestsClient/tests/browser-interop/test-results/
|
|
||||||
nestsClient/tests/browser-interop/playwright-report/
|
|
||||||
|
|
||||||
test-and-build-android:
|
test-and-build-android:
|
||||||
needs: lint
|
needs: lint
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
|
|||||||
@@ -92,9 +92,13 @@ DoD #5 (gap matrix coverage) closed.
|
|||||||
tier — Chromium's `@moq/lite` 0.2.x re-attach across
|
tier — Chromium's `@moq/lite` 0.2.x re-attach across
|
||||||
`Active::Ended → Active` is unreliable; T12 protection is
|
`Active::Ended → Active` is unreliable; T12 protection is
|
||||||
asserted by the hang-tier counterpart.
|
asserted by the hang-tier counterpart.
|
||||||
- CI gating wired in
|
- CI gating reached green-on-the-rig (10/10 × 22 = 220/220)
|
||||||
`2026-05-07-cross-stack-interop-ci-gating.md` after a 10/10
|
in commit `21947bc5`, then was deferred on cost grounds —
|
||||||
sweep × 22 tests = 220/220 pass stability bar.
|
both suites run manually now per
|
||||||
|
[`nestsClient/tests/README.md`](../tests/README.md). YAML
|
||||||
|
shape preserved in
|
||||||
|
`2026-05-07-cross-stack-interop-ci-gating.md` for the next
|
||||||
|
revisit.
|
||||||
|
|
||||||
## Files referenced
|
## Files referenced
|
||||||
|
|
||||||
|
|||||||
@@ -440,25 +440,30 @@ relay-side timing under load.
|
|||||||
|
|
||||||
## CI integration
|
## CI integration
|
||||||
|
|
||||||
**Wired** as of 2026-05-07 (commit `21947bc5`). Both
|
**Manual-run only** (deferred from CI on cost grounds). Both
|
||||||
`hang-interop` and `browser-interop` jobs in
|
suites are kept green and locally invokable; developer-facing
|
||||||
`.github/workflows/build.yml`, gated on `lint`, `ubuntu-latest`,
|
docs at [`nestsClient/tests/README.md`](../tests/README.md)
|
||||||
30 min timeout each, with cached cargo + bun + Playwright Chromium.
|
cover when/how/prerequisites.
|
||||||
|
|
||||||
Stability bar that unblocked CI: **10/10 BUILD SUCCESSFUL × 22
|
Brief history:
|
||||||
tests = 220/220 pass** on the merged branch (~5m 28s per sweep,
|
|
||||||
steady state). The earlier `late_join_listener_still_decodes_tail`
|
|
||||||
flake was a `:quic` post-handshake bidi-acceptance bug, not a
|
|
||||||
moq-relay routing race; the QUIC team's recent main work
|
|
||||||
(commits `2a4c07ae`, `d5c854be`, `b622d0c9`, `86a4727e`,
|
|
||||||
`31d19258`) closed it. See
|
|
||||||
`2026-05-07-moq-relay-routing-investigation.md` § Closure for
|
|
||||||
the trace evidence and
|
|
||||||
`2026-05-07-t16-closure-roadmap.md` for the rolled-up status.
|
|
||||||
|
|
||||||
The 2-week post-merge CI green-rate watch is on the maintainer
|
- 2026-05-07: 10/10 sweep × 22 tests = 220/220 hard-pass
|
||||||
who lands this: ≥ 95 % required per the plan's stability gate;
|
established a working stability bar after the `:quic`
|
||||||
otherwise pull both jobs and reopen the routing investigation.
|
post-handshake bidi-drop fix landed via `origin/main` (commits
|
||||||
|
`2a4c07ae`, `d5c854be`, `b622d0c9`, `86a4727e`, `31d19258`).
|
||||||
|
- Commit `21947bc5` re-added the `hang-interop` +
|
||||||
|
`browser-interop` jobs to `.github/workflows/build.yml`.
|
||||||
|
- Maintainer review then deferred CI gating on cost grounds
|
||||||
|
(cold cache ~10 min hang, ~13 min browser; most PRs don't
|
||||||
|
touch audio / MoQ / QUIC). Both jobs were removed; the YAML
|
||||||
|
shape stays preserved in the closed-but-deferred plan
|
||||||
|
[`2026-05-07-cross-stack-interop-ci-gating.md`](2026-05-07-cross-stack-interop-ci-gating.md)
|
||||||
|
for re-evaluation if the cost calculus changes.
|
||||||
|
|
||||||
|
The trace-capture instrumentation
|
||||||
|
(`-DnestsHangInteropTraceRelay=true`) added during the routing
|
||||||
|
investigation stays in place — useful when triaging a future
|
||||||
|
flake.
|
||||||
|
|
||||||
## Pending follow-ups
|
## Pending follow-ups
|
||||||
|
|
||||||
|
|||||||
@@ -1,19 +1,26 @@
|
|||||||
# Plan: wire CI gating for the cross-stack interop suite
|
# Plan: wire CI gating for the cross-stack interop suite
|
||||||
|
|
||||||
**Status:** ✅ CLOSED 2026-05-07. Both jobs wired in commit
|
**Status:** ⏸ DEFERRED 2026-05-07.
|
||||||
`21947bc5` (path-tweaked from the original removed shape per
|
The infrastructure to wire CI is ready — both jobs landed in
|
||||||
the `nestsClient/tests/browser-interop/` move). Stability-bar
|
commit `21947bc5` and a 10/10 stability sweep × 22 tests =
|
||||||
sweep: **10/10 BUILD SUCCESSFUL × 22 tests = 220/220 pass.**
|
220/220 passed on the agent rig. A maintainer review then
|
||||||
Acceptance bar from the roadmap met.
|
judged the wallclock cost too high for the change-pattern and
|
||||||
|
removed the jobs (cold cache: ~10 min hang, ~13 min browser;
|
||||||
|
most PRs don't touch audio / MoQ / QUIC).
|
||||||
|
|
||||||
|
The suites are now run manually before merging changes that
|
||||||
|
touch the relevant code paths. Developer-facing docs:
|
||||||
|
[`../tests/README.md`](../tests/README.md). This plan stays
|
||||||
|
in the tree for the next time someone wants to revisit CI
|
||||||
|
gating — the YAML shapes below are the exact reverse-merge
|
||||||
|
target, and the stability bar (10/10 sweep) has already been
|
||||||
|
demonstrated on this branch.
|
||||||
|
|
||||||
**Depended on:**
|
**Depended on:**
|
||||||
- `2026-05-07-moq-relay-routing-investigation.md` closed (✅)
|
- `2026-05-07-moq-relay-routing-investigation.md` closed (✅)
|
||||||
- `2026-05-07-tighten-cross-stack-assertions.md` closed (✅)
|
- `2026-05-07-tighten-cross-stack-assertions.md` closed (✅)
|
||||||
- 10/10 sweep stability verified (✅)
|
- 10/10 sweep stability verified (✅)
|
||||||
|
|
||||||
This is the FINAL step of the T16 closure. With stable hard-pass
|
|
||||||
suites, CI gating becomes safe and meaningful.
|
|
||||||
|
|
||||||
## What's needed
|
## What's needed
|
||||||
|
|
||||||
### A) `.github/workflows/build.yml` — the hang-interop job
|
### A) `.github/workflows/build.yml` — the hang-interop job
|
||||||
|
|||||||
@@ -67,22 +67,23 @@ HangInteropTest with their current soft-pass assertions intact.
|
|||||||
|
|
||||||
**Acceptance bar met.** 5/5 sweep with hard assertions.
|
**Acceptance bar met.** 5/5 sweep with hard assertions.
|
||||||
|
|
||||||
## Priority 3 — `2026-05-07-cross-stack-interop-ci-gating.md` ✅ CLOSED
|
## Priority 3 — `2026-05-07-cross-stack-interop-ci-gating.md` ⏸ DEFERRED (manual run only)
|
||||||
|
|
||||||
> **Closed 2026-05-07** by commit `21947bc5`. Both
|
> **Briefly wired then reverted, 2026-05-07.** Commit `21947bc5`
|
||||||
> `hang-interop` and `browser-interop` jobs landed in
|
> re-added both jobs to `build.yml`; verification gave 10/10
|
||||||
> `.github/workflows/build.yml`, gated on `lint`,
|
> sweep × 22 tests = 220/220 hard-pass on the agent rig. A
|
||||||
> `ubuntu-latest`, 30 min timeout, with cached cargo + bun +
|
> subsequent maintainer review judged the cold-cache cost
|
||||||
> Playwright Chromium. Path-tweaked from the original removed
|
> (~10 min hang, ~13 min browser) too high for the change-pattern
|
||||||
> shape because the browser harness moved from
|
> (most PRs don't touch audio / MoQ / QUIC) and removed the jobs.
|
||||||
> `nestsClient-browser-interop/` to
|
>
|
||||||
> `nestsClient/tests/browser-interop/` (commit `bd7b166f`).
|
> The suites stay green locally and are intended to run before
|
||||||
|
> merging any change that touches the audio / moq-lite / `:quic`
|
||||||
**Acceptance bar met.** 10/10 sweep BUILD SUCCESSFUL × 22 tests
|
> stack. Documentation: [`nestsClient/tests/README.md`](../tests/README.md)
|
||||||
= **220/220 pass** (~5m 28s steady state per sweep on the agent
|
> covers when/how/prerequisites/debugging.
|
||||||
rig). The first 2 weeks of post-merge CI runs still need a
|
>
|
||||||
maintainer to monitor flake rate (per the plan's "≥ 95% green
|
> The CI-gating plan is kept around in case the wallclock cost
|
||||||
rate" gate); pull the jobs again if it falls below.
|
> calculus changes (e.g., a much faster runner, or a smaller
|
||||||
|
> opt-in subset of the suite that runs in <2 min).
|
||||||
|
|
||||||
## Independent track — `2026-05-07-framespergroup-production-rerun.md`
|
## Independent track — `2026-05-07-framespergroup-production-rerun.md`
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,186 @@
|
|||||||
|
# Cross-stack interop tests
|
||||||
|
|
||||||
|
> Manually invoked. Not part of `build.yml` — see "Why not in CI" below.
|
||||||
|
|
||||||
|
Two suites that drive the production Amethyst Kotlin nests stack against
|
||||||
|
external reference implementations of moq-lite-03:
|
||||||
|
|
||||||
|
| Suite | Path | Runs | What it asserts |
|
||||||
|
|---|---|---|---|
|
||||||
|
| **`HangInteropTest`** | `nestsClient/tests/hang-interop/` | Rust `hang-listen` / `hang-publish` (kixelated reference) ↔ Amethyst Kotlin via a real `moq-relay` 0.10.x subprocess | Wire-byte capture, FFT peaks on decoded PCM, sample-count floors, mute / hot-swap / packet-loss / late-join / 60 s long broadcast / multi-listener fan-out. |
|
||||||
|
| **`BrowserInteropTest`** | `nestsClient/tests/browser-interop/` | Headless Chromium running `@moq/lite` + `@moq/hang` via Playwright ↔ Amethyst Kotlin (forward + reverse) | Same scenario family as hang-interop, plus WebCodecs `AudioDecoder` / `AudioEncoder` correctness, ALPN negotiation, browser-side reconnect. |
|
||||||
|
|
||||||
|
Coverage matrix: [`nestsClient/plans/2026-05-06-cross-stack-interop-test-gap-matrix.md`][gap]
|
||||||
|
|
||||||
|
[gap]: ../plans/2026-05-06-cross-stack-interop-test-gap-matrix.md
|
||||||
|
|
||||||
|
## When to run
|
||||||
|
|
||||||
|
Run **both suites locally before merging** any change that touches:
|
||||||
|
|
||||||
|
- `quartz/.../nip53` (audio rooms event types, catalog wire format)
|
||||||
|
- `nestsClient/src/.../moq/lite/` (moq-lite session, publisher/subscriber, codec)
|
||||||
|
- `nestsClient/src/.../audio/` (capture, encoder/decoder, broadcaster, player)
|
||||||
|
- `nestsClient/src/.../MoqLiteNestsSpeaker.kt` / `MoqLiteNestsListener.kt`
|
||||||
|
- `nestsClient/src/.../ReconnectingNests*.kt`
|
||||||
|
- `:quic` (WebTransport, packet header protection, key updates, stream demux)
|
||||||
|
- The hang/browser sidecars themselves
|
||||||
|
|
||||||
|
Skip if the change is documentation, UI-only, build-script-only, or otherwise
|
||||||
|
can't affect wire bytes / decoded audio.
|
||||||
|
|
||||||
|
## Quick start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Hang-tier (Rust ↔ Kotlin via real moq-relay subprocess)
|
||||||
|
./gradlew :nestsClient:jvmTest \
|
||||||
|
--tests "com.vitorpamplona.nestsclient.interop.native.HangInteropTest" \
|
||||||
|
-DnestsHangInterop=true
|
||||||
|
|
||||||
|
# Browser-tier (Chromium ↔ Kotlin) — also boots the moq-relay
|
||||||
|
./gradlew :nestsClient:jvmTest \
|
||||||
|
--tests "com.vitorpamplona.nestsclient.interop.native.BrowserInteropTest" \
|
||||||
|
-DnestsHangInterop=true \
|
||||||
|
-DnestsBrowserInterop=true
|
||||||
|
|
||||||
|
# Both together
|
||||||
|
./gradlew :nestsClient:jvmTest \
|
||||||
|
--tests "com.vitorpamplona.nestsclient.interop.native.HangInteropTest" \
|
||||||
|
--tests "com.vitorpamplona.nestsclient.interop.native.BrowserInteropTest" \
|
||||||
|
-DnestsHangInterop=true -DnestsBrowserInterop=true
|
||||||
|
```
|
||||||
|
|
||||||
|
The opt-in flags (`-DnestsHangInterop=true` / `-DnestsBrowserInterop=true`)
|
||||||
|
also act as JUnit `Assume` gates — without them, the suites mark every
|
||||||
|
scenario `skipped` rather than failing on missing prerequisites.
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
| Tool | Why | Install |
|
||||||
|
|---|---|---|
|
||||||
|
| **Rust ≥ 1.95** | `moq-relay` 0.10.25's transitive dep `constant_time_eq 0.4.3` requires it | `rustup install 1.95 && rustup default 1.95` |
|
||||||
|
| **`cargo`** on PATH | Builds the hang-interop sidecars + `cargo install`s `moq-relay`, `moq-token-cli` | Comes with rustup |
|
||||||
|
| **`bun` ≥ 1.3.x** (browser only) | Bundles the Chromium harness + drives Playwright | `curl -fsSL https://bun.sh/install \| bash` |
|
||||||
|
| **Playwright Chromium** (browser only) | Headless Chromium for `@moq/lite` + `@moq/hang` | Auto-installed by `interopInstallPlaywrightChromium` Gradle task |
|
||||||
|
|
||||||
|
The Gradle build automates everything from there. First run installs and
|
||||||
|
caches:
|
||||||
|
|
||||||
|
- `moq-relay` 0.10.25 + `moq-token-cli` 0.5.23 → `~/.cache/amethyst-nests-interop/hang-interop-cargo/bin/`
|
||||||
|
- Sidecar release binaries → `nestsClient/tests/hang-interop/target/release/`
|
||||||
|
- bun's `node_modules` + `dist/` → `nestsClient/tests/browser-interop/`
|
||||||
|
- Playwright Chromium → `~/.cache/ms-playwright/`
|
||||||
|
|
||||||
|
Cold first run: ~10 min hang, ~13 min browser. Cached: ~3-4 min hang,
|
||||||
|
~5-7 min browser.
|
||||||
|
|
||||||
|
## Configuration knobs
|
||||||
|
|
||||||
|
| Flag | Default | Purpose |
|
||||||
|
|---|---|---|
|
||||||
|
| `-DnestsHangInterop=true` | unset (skip) | Enable HangInteropTest. |
|
||||||
|
| `-DnestsBrowserInterop=true` | unset (skip) | Enable BrowserInteropTest. Implies `nestsHangInterop` (the browser harness boots the same `moq-relay` subprocess). |
|
||||||
|
| `-DnestsHangInteropTraceRelay=true` | unset | Per-test moq-relay trace capture. Writes the relay subprocess's combined stdout/stderr (with `RUST_LOG=info,moq_relay=trace,moq_lite=trace,moq_native=debug`) to `nestsClient/build/relay-logs/<methodName>-<seq>-<ts>.log`. Use for debugging suite flakes. |
|
||||||
|
| `-DnestsHangInteropDiagnostic=true` | unset | Runs the Kotlin↔Kotlin variant gated separately (`KotlinSpeakerKotlinListenerThroughNativeRelayTest`) — useful to bisect wire-format bugs without involving Rust or Chromium. |
|
||||||
|
| `BUN_BIN`, `NPX_BIN` (env) | autodetected | Override the bun / npx binary path if your install lives outside the agent default (`/root/.bun/bin/bun`). |
|
||||||
|
|
||||||
|
## Known limitations
|
||||||
|
|
||||||
|
- **`chromium_listener_speaker_hot_swap_does_not_crash`** soft-passes its
|
||||||
|
PCM assertion — Chromium's `@moq/lite` 0.2.x doesn't re-attach across
|
||||||
|
`Active::Ended → Active`, so the browser captures only ~100 ms post-swap.
|
||||||
|
T12 (group sequence carry across hot-swaps) is hard-asserted by the
|
||||||
|
hang-tier counterpart.
|
||||||
|
- **`framesPerGroup`** is pinned to `5` in the test scenarios. Production
|
||||||
|
defaults to `50`. The two haven't been reconciled against a single
|
||||||
|
multi-rig benchmark — see
|
||||||
|
[`framespergroup-reconciliation`](../plans/2026-05-07-framespergroup-reconciliation.md).
|
||||||
|
- **I7 cycle 2 truncation**: `moq-relay` 0.10.x truncates the second
|
||||||
|
cycle of a publisher reconnect at ~1.0 s out of ~2.5 s. Tests assert
|
||||||
|
`≥ 2.5 s` floor which the truncated cycle still clears; a future
|
||||||
|
upstream fix may let us tighten further.
|
||||||
|
- **I12 GOAWAY**: not applicable to moq-lite-03; only the IETF
|
||||||
|
moq-transport target (currently disabled) would exercise it.
|
||||||
|
|
||||||
|
Full open-issues list:
|
||||||
|
[`2026-05-06-cross-stack-interop-test-results.md` § Pending follow-ups][results].
|
||||||
|
|
||||||
|
[results]: ../plans/2026-05-06-cross-stack-interop-test-results.md
|
||||||
|
|
||||||
|
## Debugging a flaking scenario
|
||||||
|
|
||||||
|
1. Re-run the failing scenario in isolation (no `--tests` filter races):
|
||||||
|
```bash
|
||||||
|
./gradlew :nestsClient:jvmTest \
|
||||||
|
--tests "*HangInteropTest.late_join_listener_still_decodes_tail" \
|
||||||
|
-DnestsHangInterop=true \
|
||||||
|
-DnestsHangInteropTraceRelay=true \
|
||||||
|
--rerun-tasks
|
||||||
|
```
|
||||||
|
2. Inspect `nestsClient/build/relay-logs/<methodName>-*.log` for
|
||||||
|
`subscribed started` / `encoding self=Subscribe` /
|
||||||
|
`decoded result=SubscribeOk` / `subscribed cancelled` events.
|
||||||
|
3. Cross-reference with the speaker-side `Log.d("NestTx")` lines
|
||||||
|
captured in JUnit XML's `<system-err>`
|
||||||
|
(`nestsClient/build/test-results/jvmTest/TEST-*.xml`) by timestamp.
|
||||||
|
4. The fastest diagnostic loop bypasses Browser/Chromium entirely — use
|
||||||
|
the diagnostic test:
|
||||||
|
```bash
|
||||||
|
./gradlew :nestsClient:jvmTest \
|
||||||
|
--tests "*KotlinSpeakerKotlinListenerThroughNativeRelayTest" \
|
||||||
|
-DnestsHangInteropDiagnostic=true
|
||||||
|
```
|
||||||
|
|
||||||
|
Sample pre/post-merge trace pair for the previously-flaking late-join
|
||||||
|
scenario lives at
|
||||||
|
[`plans/artefacts/2026-05-07-routing-race-disproven/`](../plans/artefacts/2026-05-07-routing-race-disproven/)
|
||||||
|
+ [`plans/artefacts/2026-05-07-routing-race-closed-by-merge/`](../plans/artefacts/2026-05-07-routing-race-closed-by-merge/).
|
||||||
|
|
||||||
|
## Why not in CI
|
||||||
|
|
||||||
|
Both suites are slow on a cold cache (~10–13 min each), and even on a
|
||||||
|
warm cache the browser tier dominates the critical path at ~5-7 min.
|
||||||
|
Running them on every PR doubles CI time for changes that don't touch
|
||||||
|
audio / MoQ / QUIC.
|
||||||
|
|
||||||
|
History:
|
||||||
|
|
||||||
|
- `21947bc5` re-added both jobs after the T16 closure roadmap closed
|
||||||
|
Priority 1 (`:quic` post-handshake bidi-drop fixed via `origin/main`
|
||||||
|
merge) and Priority 2 (assertion hardening). 10/10 sweep × 22 tests
|
||||||
|
= 220/220 hard-pass on the merged branch.
|
||||||
|
- A subsequent maintainer review judged the wallclock cost too high
|
||||||
|
for the change-pattern (most PRs don't touch the relevant code) and
|
||||||
|
removed the jobs.
|
||||||
|
|
||||||
|
The suites still run locally per the rules above. If a PR DOES touch
|
||||||
|
audio / MoQ / QUIC code paths and the author hasn't run them, ask in
|
||||||
|
review.
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
- `hang-interop/REV` — pinned upstream versions (`MOQ_RELAY_VERSION`,
|
||||||
|
`MOQ_TOKEN_CLI_VERSION`, `HANG_VERSION`, `MOQ_LITE_VERSION`,
|
||||||
|
`MOQ_NATIVE_VERSION`). Bump deliberately.
|
||||||
|
- `hang-interop/Cargo.toml` + `hang-{listen,publish}/`,
|
||||||
|
`udp-loss-shim/` — Rust sidecar workspace.
|
||||||
|
- `browser-interop/package.json` + `src/` + `playwright.config.ts` —
|
||||||
|
bun + Playwright harness.
|
||||||
|
- `nestsClient/src/jvmTest/kotlin/.../interop/native/`:
|
||||||
|
- `NativeMoqRelayHarness.kt` — boots the relay subprocess + captures
|
||||||
|
per-test trace.
|
||||||
|
- `HangInteropTest.kt`, `HangInteropReverseTest.kt`,
|
||||||
|
`HangInteropMultiListenerTest.kt`,
|
||||||
|
`KotlinSpeakerKotlinListenerThroughNativeRelayTest.kt` —
|
||||||
|
hang-tier scenarios.
|
||||||
|
- `BrowserInteropTest.kt`, `PlaywrightDriver.kt` — browser-tier.
|
||||||
|
|
||||||
|
## See also
|
||||||
|
|
||||||
|
- [`2026-05-06-cross-stack-interop-test.md`](../plans/2026-05-06-cross-stack-interop-test.md)
|
||||||
|
— original spec / Definition of Done.
|
||||||
|
- [`2026-05-07-t16-closure-roadmap.md`](../plans/2026-05-07-t16-closure-roadmap.md)
|
||||||
|
— closure roadmap (Priorities 1, 2 closed; Priority 3 deferred).
|
||||||
|
- [`2026-05-07-cross-stack-interop-ci-gating.md`](../plans/2026-05-07-cross-stack-interop-ci-gating.md)
|
||||||
|
— the deferred CI-gating plan (kept around in case the wallclock
|
||||||
|
cost calculus changes).
|
||||||
Reference in New Issue
Block a user