test(nests): one-way latency + 5 % packet-loss path in audio benchmark

Expands `AudioLatencyComparisonTest` from pure inter-arrival jitter
into a real one-way latency measurement against the same
`hang-publish` Rust reference, AND adds a loss-path variant gated
through `udp-loss-shim`.

`hang-publish` (Rust sidecar):
  - new `--log-send-times` flag. When set, the publisher prints
    `SEND frame=<N> send_t_us=<UNIX_EPOCH_MICROS>` to stdout for
    every frame, captured with `SystemTime::now()` immediately
    before `frame.encode(group)`. Lock + flush per line because a
    piped Rust stdout is fully-buffered by default and the JVM-side
    parser needs a live stream. Default off so existing scenarios
    (the I7 reconnect test in particular) keep their stderr-only
    RUST_LOG output untouched.

Kotlin test:
  - `TimestampingOpusEncoder` wraps `JvmOpusEncoder` and stamps an
    `Instant.now()` epoch-micros value per `encode(...)` call into a
    shared map keyed by a monotonic frame counter. encode runs
    immediately before `publisher.send` inside
    `NestMoqLiteBroadcaster`, so the captured timestamp is the
    closest cross-stack anchor we have to "moment the frame entered
    the publisher's outbound buffer" — same anchor the Rust side
    uses.
  - One-way latency = arrival epoch-micros − send epoch-micros per
    matched frame. Both sides read `CLOCK_REALTIME` on linux/macOS,
    so values can be compared directly without a sync handshake when
    the two processes share a host.
  - Frame matching uses `MoqObject.objectId` directly. The listener
    layer already synthesises objectId as a per-SUBSCRIPTION
    monotonic counter, so it equals the publisher's absolute frame
    index; a previous draft of this test did `groupId * 5 + objectId`
    and silently aligned against the wrong frames, producing
    negative latencies.
  - New `under_5pct_packet_loss_pacing_and_one_way_latency` scenario.
    Spawns TWO `udp-loss-shim` instances (one per publisher: the
    shim latches a single client on first datagram, so a shared shim
    would silently swallow the second publisher's traffic), each
    forwarding 1:1 to the moq-relay's UDP port modulo a 5 %
    bidirectional drop rate. The listener stays on the direct path
    so any latency growth comes from publisher-side retransmit /
    ack feedback, not listener-side loss.
  - Both Kotlin and Rust pinned to `FRAMES_PER_GROUP = 5` (Rust's
    default; Kotlin's default is 50). Matched so the per-group
    uni-stream open/close cost is the same on both sides — the
    comparison is about implementation, not about which stack picked
    which group size.

Observed (10 s window, localhost, MacBook Pro M2):

  ===== Audio publisher comparison: clean (loss=0%) =====
  Kotlin speaker     ttf=137 ms  inter-arrival p50/95/99/max=20.17/20.67/20.88/21.16 ms
                                  one-way        p50/95/99/max=80.60/81.31/81.74/83.91 ms
  Rust hang-publish  ttf=325 ms  inter-arrival p50/95/99/max=20.00/21.18/21.49/21.95 ms
                                  one-way        p50/95/99/max=100.32/101.45/102.04/168.58 ms
  ===== Audio publisher comparison: loss-5pct (loss=5%) =====
  Kotlin speaker                  one-way        p50/95/99/max=80.50/81.55/82.31/86.43 ms
  Rust hang-publish               one-way        p50/95/99/max=100.44/119.88/122.16/158.98 ms

  Reading: Kotlin matches Rust on the clean path within ~20 ms
  baseline (and actually sits below it), and absorbs 5 % loss with
  only +0.6 ms p99 growth vs Rust's +20 ms. The 80-100 ms baseline
  is moq-lite's group-buffer floor with 5-frames/group — protocol,
  not stack.

Asserts: each side delivers >= 80 % of expected frames on the clean
path, >= 60 % under loss, and clean-path median inter-arrival sits
in [15, 35] ms. Tail percentiles are printed, not gated.

Gated by `-DnestsHangInterop=true`; both sidecars (hang-publish,
udp-loss-shim) come from `interopBuildSidecars`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Vitor Pamplona
2026-05-15 09:27:25 -04:00
parent e18e6c8399
commit 9fd8cddebf
2 changed files with 610 additions and 258 deletions
@@ -7,7 +7,8 @@
//! the Rust → Amethyst direction. See
//! `nestsClient/plans/2026-05-06-cross-stack-interop-test.md`.
use std::time::Duration;
use std::io::Write;
use std::time::{Duration, SystemTime, UNIX_EPOCH};
use anyhow::{Context, anyhow};
use bytes::Bytes;
@@ -87,6 +88,25 @@ struct Args {
/// exercise the Kotlin listener's publisher-cycle handling.
#[arg(long, default_value_t = 0)]
reconnect_after_ms: u64,
/// If set, emit one line per frame to stdout, immediately before
/// the frame is handed to the moq-lite group producer:
///
/// `SEND frame=<N> send_t_us=<UNIX_EPOCH_MICROS>`
///
/// Used by `AudioLatencyComparisonTest` to pair publisher send
/// time with the listener's arrival time and compute one-way
/// latency. `send_t_us` is the wall-clock microseconds since
/// `UNIX_EPOCH` (CLOCK_REALTIME on linux / macOS), which matches
/// what the JVM's `Instant.now()` reads on the listener side —
/// the two clocks share the same source when both processes run
/// on the same host.
///
/// Disabled by default so existing interop scenarios (I7, etc.)
/// keep their plain stderr-only RUST_LOG output and don't have
/// to filter through 50 frame-tag lines per second on stdout.
#[arg(long, default_value_t = false)]
log_send_times: bool,
}
#[tokio::main]
@@ -364,6 +384,37 @@ async fn publish_cycle(
}
let g = group.as_mut().expect("group always Some after init");
if args.log_send_times {
// Read SystemTime AS LATE AS POSSIBLE before the moq-lite
// group write so the captured `send_t_us` reflects "moment
// the frame entered the publisher's outbound buffer". The
// group write is non-blocking (the per-stream send queue
// does the actual QUIC work asynchronously) so this is the
// closest deterministic anchor we have to "send time" from
// application code. Unwrap on the duration_since: the
// system clock running before UNIX_EPOCH would be a
// catastrophic environment failure, not a test condition.
let send_t_us = SystemTime::now()
.duration_since(UNIX_EPOCH)
.expect("system clock predates UNIX_EPOCH")
.as_micros();
// Bypass tracing — RUST_LOG goes to stderr by config and
// the test parses stdout. Lock + flush per line because
// Rust's `Stdout` is fully-buffered when not a TTY (which
// is exactly our case here — `redirectErrorStream(true)`
// on the JVM side makes stdout a pipe), so a passive
// `writeln!` would batch frames into ~8 KB chunks and the
// parser would see them in bursts instead of as a live
// stream. Per-line flush is cheap (≤ 50 lines/s/publisher).
let stdout = std::io::stdout();
let mut handle = stdout.lock();
let _ = writeln!(
handle,
"SEND frame={} send_t_us={}",
*frame_no, send_t_us
);
let _ = handle.flush();
}
frame.encode(g).context("encode hang frame into group")?;
frames_in_group += 1;
if frames_in_group == FRAMES_PER_GROUP {