amethyst/nestsClient/plans/2026-05-06-cross-stack-interop-test-results.md

Plan: cross-stack interop test (T16) — Phase 1 + Phase 2 results

Status: Phase 1 + most of Phase 2 landed. Phases 3–5 still deferred.

Phase 2 update

Added on top of the Phase 1 scaffolding:

• hang-listen real body — connects to a moq-lite-03 relay, reads the hang catalog, picks the first Opus / Container::Legacy audio rendition, decodes each Opus packet via the opus = "0.3" crate, and writes Float32 little-endian PCM to --output-pcm.
• hang-publish real body — claims a broadcast, publishes a hang catalog with one Opus rendition (track name configurable via --track-name, default audio/data to match Amethyst's MoqLiteNestsListener.AUDIO_TRACK), encodes a sine wave with libopus, and pumps Opus frames in 5-frame groups for --duration seconds. Uses audiopus-equivalent opus = "0.3".
• Both binaries explicitly install the rustls aws-lc-rs crypto provider (rustls 0.23 no longer auto-installs) and use --client-version moq-lite-03 + --tls-disable-verify=true to interop with the harness's self-signed --tls-generate localhost relay.
• JVM Opus encoder/decoder via club.minnced:opus-java:1.1.1 (JNA bindings + bundled libopus.so / libopus.dylib / opus.dll natives). Lives in nestsClient/src/jvmTest/.../audio/JvmOpusEncoder.kt
  • JvmOpusDecoder.kt. Verified by JvmOpusRoundTripTest.sine_440_round_trips_through_libopus — encode → decode preserves the FFT peak at 440 Hz and the zero-crossing rate at 880/sec within 5%.
• Real-time pacing in SineWaveAudioCapture — readFrame blocks until the next 20-ms boundary, mirroring how a microphone source paces. Without this the broadcaster's read loop would flood the relay with millions of frames/sec.
• HangInteropTest.rust_hang_publish_to_rust_hang_listener_round_trip_440 — Rust↔Rust round-trip through the harness. Spawns hang-publish
  • hang-listen as subprocesses, asserts the decoded PCM has FFT peak at 440 Hz, ZCR at 880/sec, and 5 s of samples (±20% slack for Opus look-ahead + relay buffering). Verified green on Linux x86_64.

I1 — Amethyst speaker → hang-listen — green at framesPerGroup=5

Initial diagnosis (Kotlin speaker → hang-listen sees Group { subscribe, sequence } headers but no frame payloads) was bisected by adding KotlinSpeakerKotlinListenerThroughNativeRelayTest — a Kotlin↔Kotlin path through the same moq-relay 0.10.x. That test reproduced the failure too, ruling out a Kotlin↔Rust-specific mismatch. Bisecting framesPerGroup:

• framesPerGroup = 1 (one frame per uni stream): passes
• framesPerGroup = 5 (the value nestsClient/plans/2026-05-01-quic-stream-cliff-investigation.md recommends): passes
• framesPerGroup = 50 (the repo's current NestMoqLiteBroadcaster.DEFAULT_FRAMES_PER_GROUP): fails

The 50-frame default writes ~6 KB onto a single uni stream over ~1 s, which exceeds moq-relay 0.10.25's per-subscriber forward buffer; the relay forwards the Group control header but holds the frame data, never delivering it downstream. This matches the audit summarised in the cliff-investigation plan: the bug is a moq-relay 0.10.x policy interacting with our publish cadence, not a wire-format defect on either side.

HangInteropTest.amethyst_speaker_to_hang_listener_static_tone_440 now pins framesPerGroup = 5 to match what the cliff plan already documents as the safe production cadence. The Kotlin↔ Kotlin diagnostic test KotlinSpeakerKotlinListenerThroughNativeRelayTest also pins framesPerGroup = 5 and is kept as a regression for the cadence interaction — if a future relay bump changes the ceiling, both tests will trip together and the failure will be attributable in one place.

Follow-up (out of scope here): the repo's NestMoqLiteBroadcaster.DEFAULT_FRAMES_PER_GROUP = 50 should move down to 5 to match the cliff plan and the values our production deployment already uses on the wire. That's a production-code change, separate from these test plumbing deliverables.

Origin: companion to nestsClient/plans/2026-05-06-cross-stack-interop-test.md. This file records what actually shipped in Phase 1, the deviations from the spec, and the concrete pickup points for Phase 2.

What landed

Cargo workspace (cli/hang-interop/)

• Workspace with three binary crates: hang-listen, hang-publish, udp-loss-shim. All three are Phase-1 stubs — they parse their CLI flags via clap, print a banner, and exit 0. Phase 2 fills the bodies.
• cli/hang-interop/REV documents the pinned upstream kixelated/moq rev (9e2461ee...) plus the published crate versions on crates.io that track that rev (moq-relay 0.10.25, moq-token-cli 0.5.23, hang 0.15.8, moq-lite 0.15.15, moq-native 0.13).
• cargo build --release is verified green from the workspace root.

Gradle integration (nestsClient/build.gradle.kts)

• interopInstallMoqRelay — cargo install moq-relay --version $moqRelayVersion --root <cache> into ~/.cache/amethyst-nests-interop/hang-interop-cargo/. Skips when the binary already exists in the cache.
• interopInstallMoqTokenCli — same shape for moq-token-cli.
• interopBuildSidecars — cargo build --release over the local cli/hang-interop/ workspace.
• interopBuildHangSidecars — umbrella task that depends on the three above. Runs as a test dependency only when -DnestsHangInterop=true is set.
• Test-task wiring forwards nestsHangInteropSidecarsDir and nestsHangInteropCargoBinDir system properties so the harness can find the binaries. nestsHangInterop itself is also forwarded — mirrors the existing nestsInterop opt-in.

Kotlin harness (nestsClient/src/jvmTest/...)

• interop/native/NativeMoqRelayHarness.kt — boots a real moq-relay subprocess with --tls-generate localhost (relay self-signs its cert at startup) and --auth-public "" (every path is treated as public, no JWT required). Uses ServerSocket(0) to reserve an ephemeral port. Tracks process output in a 64-line ring buffer so a startup failure includes the relay's stderr tail. Singleton-per-JVM via shared(), shutdown via JVM hook, mirroring the existing NostrNestsHarness pattern. Public surface: relayUrl, loopbackHostPort(), hangListenBin(), hangPublishBin(), udpLossShimBin(), moqTokenBin().
• audio/SineWaveAudioCapture.kt — frame-perfect deterministic sine wave at any frequency, mono, conforming to the existing AudioCapture interface in commonMain.
• audio/PcmAssertions.kt — pure-Kotlin signal-domain assertions: assertSampleCount, assertRms, assertFftPeak (Hann-windowed iterative Cooley-Tukey, ~100 lines, no transform-library dep), assertZeroCrossingRate, findSilenceWindow. The plan spec'd these for Phase 1; everything is in commonTest now and ready for the Phase 2 scenarios.
• interop/native/NativeMoqRelayHarnessSmokeTest.kt — the only test that runs in Phase 1. Boots the harness, verifies the relay binds its UDP port, verifies the sidecar binaries are executable and the hang-listen stub runs cleanly. Does not assert any wire format — that's Phase 2.

Deviations from the spec

Plan	Reality	Why
In-process Kotlin JWT minter (ES256, JWKS file mounted into relay)	Relay configured with --auth-public ""; no JWT required	The plan flagged JWT issuance as "implementation detail" — wire-format and protocol assertions don't need real auth, and --auth-public short-circuits the whole minter + JWKS file dance. JWT validation as an interop concern is already covered by the existing NostrNestsAuthInteropTest against the Docker'd moq-auth. The cargo-installed moq-token CLI is exposed via moqTokenBin() for Phase 2 scenarios that DO want to mint a real token (e.g. revocation, expiry, kid-mismatch).
Self-signed cert generated at suite setup via Bouncy Castle / openssl req -x509	moq-relay --tls-generate localhost (relay self-signs at startup)	moq-native ships this behaviour; saves us writing PEM I/O + cert generation. The Kotlin client can use the existing PermissiveCertValidator to skip chain validation.
relay.toml config file	CLI flags only	Same effect, fewer moving parts. Field names in the plan (server.listen, tls.cert/key, auth.jwks_path) were speculative; actual moq-native flags are --server-bind, --tls-cert/--tls-key/--tls-generate, --auth-key/--auth-public.
Build moq-relay from a pinned kixelated/moq checkout via cargo build --release -p moq-relay	cargo install moq-relay --version 0.10.25 --root <cache>	moq-relay and moq-token-cli are published on crates.io; cargo install makes the install reproducible without embedding/cloning the upstream workspace. Cache key is the pinned version.
Phase 1 step 7: "Wire one passing test: I1, A→hang"	Smoke test only — no wire-format scenario	I1 needs a JVM-side Opus encoder (the OpusEncoder interface in commonMain only has an Android MediaCodec actual today), AND it needs hang-listen to actually subscribe to a moq-lite session and decode Opus to PCM. Both are Phase 2 work. The smoke test we landed proves all the harness load-bearing pieces work, so Phase 2 only has to fill in the codecs + the sidecar bodies.

Phase 2 pickup

The core gap: JVM Opus encoder + decoder, plus filling the sidecar binaries. Everything else is in place.

Step A — JVM Opus encoder/decoder

Add a JVM actual for OpusEncoder / OpusDecoder (currently Android-only via MediaCodecOpusEncoder / MediaCodecOpusDecoder). Two viable options:

1. org.concentus:Concentus (pure-Java port of libopus). On Maven Central. License-compatible. Slower than native libopus but plenty fast enough for a 48 kHz mono test stream. Recommended since it adds zero native dependency.
2. audiopus_jni (vendored or via JitPack) — wraps the same audiopus Rust crate the upstream hang examples use. Faster but adds a JNI shared object per platform.

Wire it into nestsClient/src/jvmMain/.../audio/JvmOpusEncoder.kt + JvmOpusDecoder.kt, with the Android MediaCodec actuals unchanged. Add CapturingOpusDecoder (the plan's Phase 1 ask) once the JVM decoder exists.

Step B — Fill hang-listen

Model on kixelated/moq/rs/hang/examples/subscribe.rs (the subscribe example in the upstream workspace at /tmp/moq/rs/hang/examples/subscribe.rs if recloned). Replace its video-track logic with the audio path: pick the first rendition with container.kind == "legacy" and codec == "opus", subscribe, decode each Frame into a Bytes-encoded VarInt timestamp + Opus packet, run the Opus packets through audiopus::Decoder, write Float32 PCM to --output-pcm. Dependencies to add to cli/hang-interop/hang-listen/Cargo.toml:

hang = "0.15"
moq-lite = "0.15"
moq-mux = "0.3"
moq-native = { version = "0.13", default-features = false, features = ["quinn", "aws-lc-rs"] }
web-transport-quinn = "0.11"
audiopus = "0.3"
bytes = "1"
tracing = "0.1"

Step C — Fill hang-publish

Mirror of hang-listen for the reverse direction. Generate a sine wave in Rust, encode with audiopus::Encoder, publish a hang catalog with one Opus rendition, push frames as varint(timestamp_us) + opus_packet per the hang::container::Frame::encode contract (see /tmp/moq/rs/hang/src/container/frame.rs).

Step D — Wire the I1 scenario

Once A/B/C land, the I1 "amethyst speaker → hang listener" test is straightforward — see the spec for the pattern. The harness + SineWaveAudioCapture + PcmAssertions are already in place to support it.

Phase 3 + 4 + 5 deferred

Untouched in Phase 1:

• Phase 3 transport robustness (udp-loss-shim body, hot-swap, long-broadcast).
• Phase 4 browser harness (nestsClient-browser-interop/ directory, Playwright driver).
• Phase 5 browser-only scenarios.

CI integration (the GitHub Actions workflow updates the spec shows) is also pending — until Phase 2 lands a real test, there's nothing in -DnestsHangInterop=true worth gating CI on except the smoke test.

Files added in Phase 1

cli/hang-interop/
├── REV
├── Cargo.toml
├── hang-listen/{Cargo.toml,src/main.rs}
├── hang-publish/{Cargo.toml,src/main.rs}
└── udp-loss-shim/{Cargo.toml,src/main.rs}

nestsClient/build.gradle.kts                  # +interopBuildHangSidecars + system props
nestsClient/src/jvmTest/kotlin/com/vitorpamplona/nestsclient/
├── audio/
│   ├── PcmAssertions.kt
│   └── SineWaveAudioCapture.kt
└── interop/native/
    ├── NativeMoqRelayHarness.kt
    └── NativeMoqRelayHarnessSmokeTest.kt

nestsClient/plans/2026-05-06-cross-stack-interop-test-results.md  # this file