Files

T

History

Vitor Pamplona f512478bcc test(nests): sweep packet-loss rates + Linux docker harness for latency bench

`AudioLatencyComparisonTest`:
  - Three new scenarios: 10 %, 20 %, 30 % loss in addition to the
    existing clean and 5 %. The whole benchmark now sweeps a useful
    arc for finding the breakdown point.
  - Frame-count floor now scales with loss instead of a fixed 60 %
    threshold. QUIC streams are reliable so loss surfaces mainly as
    latency, but at high rates the retransmit + congestion-control
    feedback can eat into the 10 s window, so the floor relaxes to
    `max(50, (1 - 3*lossRate) * EXPECTED_FRAMES)`. The 50-frame
    absolute floor (= 1 s of audio) catches a catastrophic publisher
    failure without flapping on the loaded loss-path runs.

Linux benchmark harness (`nestsClient/tests/hang-interop/linux-bench/`):
  - `Dockerfile` for an arm64/amd64 image with JDK 21 + rustc 1.95 +
    cmake + the *-sys-crate build deps (libssl-dev, build-essential,
    pkg-config). Pins rustc 1.95 explicitly so moq-relay v0.10.25's
    Cargo.lock-requested `constant_time_eq@0.4.3` builds cleanly.
  - `run.sh` wrapper that:
    * builds the image,
    * unpacks a filtered tarball of the host repo into the container
      (mac host `nestsClient/build/` and `cargo target/` stay
      untouched),
    * symlinks `nestsClient/build` -> `/persistent-gradle-build` and
      `nestsClient/tests/hang-interop/target` ->
      `/persistent-cargo-target` so successive runs reuse compiled
      artefacts (~3-5 min cached vs ~7-15 min cold),
    * runs the gradle test invocation and surfaces the JUnit XML
      stats to `linux-bench/out/`.

Linux numbers observed (arm64 Docker on a MacBook M2, 10 s window):

  | Loss | Stack  | p50   | p95   | p99   | max   | Frames |
  |------|--------|-------|-------|-------|-------|--------|
  |  0 % | Kotlin |  61.1 |  63.1 |  65.5 |  79.1 |   573  |
  |  0 % | Rust   | 100.7 | 102.7 | 104.9 | 140.2 |   495  |
  |  5 % | Kotlin | 380.9 | 383.4 | 385.7 | 390.5 |   556  |
  |  5 % | Rust   |   0.7 |   7.3 |  23.2 |  84.2 |   500  |
  | 10 % | Kotlin | 120.5 | 122.4 | 124.0 | 132.6 |   568  |
  | 10 % | Rust   |   0.5 |  21.9 |  60.4 |  84.4 |   500  |
  | 20 % | Kotlin | 159.7 | 164.3 | 170.3 | 183.1 |   566  |
  | 20 % | Rust   |   1.6 |  61.3 | 101.9 | 144.9 |   499  |
  | 30 % | Kotlin | 181.0 | 184.2 | 188.1 | 194.0 |   566  |
  | 30 % | Rust   |   1.2 |  80.7 | 105.3 | 140.8 |   491  |

Notes for future readers: Rust's p50 collapsing to ~1 ms under loss
is the SUBSCRIBE-late-arrival signal, not "Rust is 100x faster" — by
the time the SUBSCRIBE lands at the publisher, the in-flight
encoder has moved past those early frames, so the listener starts
receiving in near-real-time. Read p95/p99/max for the per-frame
story (Rust expands 7 -> 22 -> 61 -> 80 ms p95 across loss rates).
Frame delivery is stable for both stacks even at 30 % loss on Linux,
unlike the macOS run which hits a 3.4 s Rust catastrophe at 30 %.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-15 13:01:55 -04:00

browser-interop

chore(nests): mv nestsClient-browser-interop/ → nestsClient/tests/browser-interop/

2026-05-07 14:43:55 +00:00

hang-interop

test(nests): sweep packet-loss rates + Linux docker harness for latency bench

2026-05-15 13:01:55 -04:00

README.md

ci(nests): defer cross-stack interop CI; document manual run

2026-05-07 23:18:31 +00:00

README.md

Cross-stack interop tests

Manually invoked. Not part of build.yml — see "Why not in CI" below.

Two suites that drive the production Amethyst Kotlin nests stack against external reference implementations of moq-lite-03:

Suite	Path	Runs	What it asserts
`HangInteropTest`	`nestsClient/tests/hang-interop/`	Rust `hang-listen` / `hang-publish` (kixelated reference) ↔ Amethyst Kotlin via a real `moq-relay` 0.10.x subprocess	Wire-byte capture, FFT peaks on decoded PCM, sample-count floors, mute / hot-swap / packet-loss / late-join / 60 s long broadcast / multi-listener fan-out.
`BrowserInteropTest`	`nestsClient/tests/browser-interop/`	Headless Chromium running `@moq/lite` + `@moq/hang` via Playwright ↔ Amethyst Kotlin (forward + reverse)	Same scenario family as hang-interop, plus WebCodecs `AudioDecoder` / `AudioEncoder` correctness, ALPN negotiation, browser-side reconnect.

Coverage matrix: nestsClient/plans/2026-05-06-cross-stack-interop-test-gap-matrix.md

When to run

Run both suites locally before merging any change that touches:

quartz/.../nip53 (audio rooms event types, catalog wire format)
nestsClient/src/.../moq/lite/ (moq-lite session, publisher/subscriber, codec)
nestsClient/src/.../audio/ (capture, encoder/decoder, broadcaster, player)
nestsClient/src/.../MoqLiteNestsSpeaker.kt / MoqLiteNestsListener.kt
nestsClient/src/.../ReconnectingNests*.kt
:quic (WebTransport, packet header protection, key updates, stream demux)
The hang/browser sidecars themselves

Skip if the change is documentation, UI-only, build-script-only, or otherwise can't affect wire bytes / decoded audio.

Quick start

# Hang-tier (Rust ↔ Kotlin via real moq-relay subprocess)
./gradlew :nestsClient:jvmTest \
    --tests "com.vitorpamplona.nestsclient.interop.native.HangInteropTest" \
    -DnestsHangInterop=true

# Browser-tier (Chromium ↔ Kotlin) — also boots the moq-relay
./gradlew :nestsClient:jvmTest \
    --tests "com.vitorpamplona.nestsclient.interop.native.BrowserInteropTest" \
    -DnestsHangInterop=true \
    -DnestsBrowserInterop=true

# Both together
./gradlew :nestsClient:jvmTest \
    --tests "com.vitorpamplona.nestsclient.interop.native.HangInteropTest" \
    --tests "com.vitorpamplona.nestsclient.interop.native.BrowserInteropTest" \
    -DnestsHangInterop=true -DnestsBrowserInterop=true

The opt-in flags (-DnestsHangInterop=true / -DnestsBrowserInterop=true) also act as JUnit Assume gates — without them, the suites mark every scenario skipped rather than failing on missing prerequisites.

Prerequisites

Tool	Why	Install
Rust ≥ 1.95	`moq-relay` 0.10.25's transitive dep `constant_time_eq 0.4.3` requires it	`rustup install 1.95 && rustup default 1.95`
`cargo` on PATH	Builds the hang-interop sidecars + `cargo install`s `moq-relay`, `moq-token-cli`	Comes with rustup
`bun` ≥ 1.3.x (browser only)	Bundles the Chromium harness + drives Playwright	`curl -fsSL https://bun.sh/install \| bash`
Playwright Chromium (browser only)	Headless Chromium for `@moq/lite` + `@moq/hang`	Auto-installed by `interopInstallPlaywrightChromium` Gradle task

The Gradle build automates everything from there. First run installs and caches:

moq-relay 0.10.25 + moq-token-cli 0.5.23 → ~/.cache/amethyst-nests-interop/hang-interop-cargo/bin/
Sidecar release binaries → nestsClient/tests/hang-interop/target/release/
bun's node_modules + dist/ → nestsClient/tests/browser-interop/
Playwright Chromium → ~/.cache/ms-playwright/

Cold first run: ~10 min hang, ~13 min browser. Cached: ~3-4 min hang, ~5-7 min browser.

Configuration knobs

Flag	Default	Purpose
`-DnestsHangInterop=true`	unset (skip)	Enable HangInteropTest.
`-DnestsBrowserInterop=true`	unset (skip)	Enable BrowserInteropTest. Implies `nestsHangInterop` (the browser harness boots the same `moq-relay` subprocess).
`-DnestsHangInteropTraceRelay=true`	unset	Per-test moq-relay trace capture. Writes the relay subprocess's combined stdout/stderr (with `RUST_LOG=info,moq_relay=trace,moq_lite=trace,moq_native=debug`) to `nestsClient/build/relay-logs/<methodName>-<seq>-<ts>.log`. Use for debugging suite flakes.
`-DnestsHangInteropDiagnostic=true`	unset	Runs the Kotlin↔Kotlin variant gated separately (`KotlinSpeakerKotlinListenerThroughNativeRelayTest`) — useful to bisect wire-format bugs without involving Rust or Chromium.
`BUN_BIN`, `NPX_BIN` (env)	autodetected	Override the bun / npx binary path if your install lives outside the agent default (`/root/.bun/bin/bun`).

Known limitations

chromium_listener_speaker_hot_swap_does_not_crash soft-passes its PCM assertion — Chromium's @moq/lite 0.2.x doesn't re-attach across Active::Ended → Active, so the browser captures only ~100 ms post-swap. T12 (group sequence carry across hot-swaps) is hard-asserted by the hang-tier counterpart.
framesPerGroup is pinned to 5 in the test scenarios. Production defaults to 50. The two haven't been reconciled against a single multi-rig benchmark — see framespergroup-reconciliation.
I7 cycle 2 truncation: moq-relay 0.10.x truncates the second cycle of a publisher reconnect at ~1.0 s out of ~2.5 s. Tests assert ≥ 2.5 s floor which the truncated cycle still clears; a future upstream fix may let us tighten further.
I12 GOAWAY: not applicable to moq-lite-03; only the IETF moq-transport target (currently disabled) would exercise it.

Full open-issues list: 2026-05-06-cross-stack-interop-test-results.md § Pending follow-ups.

Debugging a flaking scenario

Re-run the failing scenario in isolation (no --tests filter races):

./gradlew :nestsClient:jvmTest \
    --tests "*HangInteropTest.late_join_listener_still_decodes_tail" \
    -DnestsHangInterop=true \
    -DnestsHangInteropTraceRelay=true \
    --rerun-tasks

Inspect nestsClient/build/relay-logs/<methodName>-*.log for subscribed started / encoding self=Subscribe / decoded result=SubscribeOk / subscribed cancelled events.
Cross-reference with the speaker-side Log.d("NestTx") lines captured in JUnit XML's <system-err> (nestsClient/build/test-results/jvmTest/TEST-*.xml) by timestamp.

The fastest diagnostic loop bypasses Browser/Chromium entirely — use the diagnostic test:

./gradlew :nestsClient:jvmTest \
    --tests "*KotlinSpeakerKotlinListenerThroughNativeRelayTest" \
    -DnestsHangInteropDiagnostic=true

Sample pre/post-merge trace pair for the previously-flaking late-join scenario lives at plans/artefacts/2026-05-07-routing-race-disproven/

plans/artefacts/2026-05-07-routing-race-closed-by-merge/.

Why not in CI

Both suites are slow on a cold cache (~10–13 min each), and even on a warm cache the browser tier dominates the critical path at ~5-7 min. Running them on every PR doubles CI time for changes that don't touch audio / MoQ / QUIC.

History:

21947bc5 re-added both jobs after the T16 closure roadmap closed Priority 1 (:quic post-handshake bidi-drop fixed via origin/main merge) and Priority 2 (assertion hardening). 10/10 sweep × 22 tests = 220/220 hard-pass on the merged branch.
A subsequent maintainer review judged the wallclock cost too high for the change-pattern (most PRs don't touch the relevant code) and removed the jobs.

The suites still run locally per the rules above. If a PR DOES touch audio / MoQ / QUIC code paths and the author hasn't run them, ask in review.

Files

hang-interop/REV — pinned upstream versions (MOQ_RELAY_VERSION, MOQ_TOKEN_CLI_VERSION, HANG_VERSION, MOQ_LITE_VERSION, MOQ_NATIVE_VERSION). Bump deliberately.
hang-interop/Cargo.toml + hang-{listen,publish}/, udp-loss-shim/ — Rust sidecar workspace.
browser-interop/package.json + src/ + playwright.config.ts — bun + Playwright harness.
nestsClient/src/jvmTest/kotlin/.../interop/native/:
- NativeMoqRelayHarness.kt — boots the relay subprocess + captures per-test trace.
- HangInteropTest.kt, HangInteropReverseTest.kt, HangInteropMultiListenerTest.kt, KotlinSpeakerKotlinListenerThroughNativeRelayTest.kt — hang-tier scenarios.
- BrowserInteropTest.kt, PlaywrightDriver.kt — browser-tier.

README.md Unescape Escape

Cross-stack interop tests

When to run

Quick start

Prerequisites

Configuration knobs

Known limitations

Debugging a flaking scenario

Why not in CI

Files

See also

README.md