docs(quic-interop): add quinn to the default peer set
quinn is now treated as a flawless-required interop target alongside aioquic, picoquic, and quic-go. Most Rust-based Nostr/MoQ relays our users run their servers on are built on quinn, so an interop regression there is a user-visible regression. Validated by a 3-round flakiness sweep (1 full matrix + 2 audio-critical subsets) — zero result flakiness across 528 test executions across all four peers, with two environmental docker-compose stall classes documented separately. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -196,6 +196,49 @@ After all of these, expectation is aioquic / picoquic ≥6/7 (M
|
||||
might still be flaky). retry green via qlog-driven debugging is
|
||||
the marquee result.
|
||||
|
||||
## Phase 6 — landed 2026-05-10 (matrix expansion + flakiness sweep)
|
||||
|
||||
Three-round flakiness sweep across four peers (aioquic, picoquic,
|
||||
quic-go, quinn). Round 1 = full 24-testcase matrix; rounds 2 and 3 =
|
||||
11-test audio-critical subset (handshake, transfer, multiplexing,
|
||||
transferloss, transfercorruption, handshakeloss, handshakecorruption,
|
||||
longrtt, blackhole, keyupdate, retry).
|
||||
|
||||
**quinn added** to the default set this phase — was previously
|
||||
untested; cohort coverage demanded it because most Rust-based Nostr /
|
||||
MoQ relays use it on the server side. Validated identical to the
|
||||
existing three:
|
||||
|
||||
| Peer | Pass | Unsupported | Fail | Goodput (kbps) | Crosstraffic (kbps) |
|
||||
|---|---|---|---|---|---|
|
||||
| aioquic | 19/22 | E, CM, V2 | — | 9071 ± 53 | 5084 ± 257 |
|
||||
| picoquic | 20/22 | V2 | CM (expected — no client-initiated migration) | 9234 ± 12 | 7405 ± 140 |
|
||||
| quic-go | 19/22 | E, CM, V2 | — | 9449 ± 8 | 6276 ± 218 |
|
||||
| quinn | 20/22 | CM, V2 | — | 9322 ± 6 | 4117 ± 2283 (env-inflated; see below) |
|
||||
|
||||
Across **528 test executions** (4 peers × 1 full round + 4 peers × 2
|
||||
audio-critical rounds × 11 tests, plus 10-iter measurements):
|
||||
- **Zero result flakiness.** No testcase that passed on one round ever
|
||||
failed on another, and vice versa.
|
||||
- **Two stall classes**, both environmental (Docker Desktop on macOS,
|
||||
not in `:quic`):
|
||||
- **Type A (compose-init lockup)** — `docker compose up` hangs for
|
||||
~15-16 min between Python's command-log and compose's first Docker
|
||||
API call. Hit 4 testcases: picoquic r2 transfer, quic-go r2
|
||||
handshakecorruption, quinn r1 rebind-port/rebind-addr (both),
|
||||
quinn r3 handshakeloss. Test always succeeds once compose starts.
|
||||
qlog confirms the QUIC connection itself completes in 10-30s. The
|
||||
quinn-r1 crosstraffic std-dev (±2283 kbps) is entirely driven by
|
||||
2 of 5 iterations being trapped in this stall — the 3 unaffected
|
||||
iters were 111/138/97 s = consistent with peers.
|
||||
- **Type B (pyshark XML stall)** — runner's `_get_packets()`
|
||||
iteration through tshark's XML via pyshark takes 16+ min on
|
||||
rebind/crosstraffic pcaps. Direct tshark CLI on the same pcap:
|
||||
0.6 s. pyshark wrapper: 66 s. Multiple passes per testcase
|
||||
amplify it.
|
||||
- Mitigations documented in `plans/2026-05-06-interop-runner.md`
|
||||
Docker Desktop section (separate investigation note).
|
||||
|
||||
## Still open
|
||||
|
||||
- **`v2`** — server demands QUIC v2 (RFC 9369). We're v1-only.
|
||||
@@ -233,11 +276,23 @@ docker-compose.yml hardcodes `container_name: sim/server/client` —
|
||||
Docker enforces those globally. Use a sequential loop:
|
||||
|
||||
```
|
||||
for peer in aioquic picoquic quic-go; do
|
||||
for peer in aioquic picoquic quic-go quinn; do
|
||||
quic/interop/run-matrix.sh -s $peer -t handshake,chacha20,...
|
||||
done
|
||||
```
|
||||
|
||||
## Default peer set
|
||||
|
||||
The four-peer baseline that every change touching `:quic` should be
|
||||
checked against is `aioquic`, `picoquic`, `quic-go`, and `quinn`. The
|
||||
rationale is product-shaped, not protocol-shaped: this is the cohort
|
||||
that the Nostr / MoQ relay ecosystem our users actually run their
|
||||
servers on is built on. quinn in particular underwrites most of the
|
||||
Rust relay stacks, so an interop regression there is a user-visible
|
||||
regression — it must stay flawless. quinn was added to the default
|
||||
set 2026-05-10 after running cleanly across the same audio-critical
|
||||
3-round flakiness sweep used for the other three peers.
|
||||
|
||||
## Explicitly unsupported testcases (return 127, runner skips)
|
||||
|
||||
| Testcase | Reason |
|
||||
|
||||
@@ -10,6 +10,19 @@
|
||||
# quic/interop/run-matrix.sh -s aioquic -t handshake # one test
|
||||
# quic/interop/run-matrix.sh -s quic-go -t handshake,chacha20 # different peer
|
||||
#
|
||||
# Default peer set when running the full nightly sweep:
|
||||
# aioquic, picoquic, quic-go, quinn — these are the four reference impls
|
||||
# we hold to "must be flawless" because the Nostr/MoQ relay ecosystem
|
||||
# our users actually run their servers on overlaps with all four
|
||||
# (notably quinn: most Rust-based relays are built on it). Always sweep
|
||||
# sequentially — run-matrix.sh is not safe to invoke concurrently (the
|
||||
# runner's docker-compose.yml hardcodes container_name: sim/server/client,
|
||||
# so parallel invocations collide).
|
||||
#
|
||||
# for peer in aioquic picoquic quic-go quinn; do
|
||||
# quic/interop/run-matrix.sh -s $peer
|
||||
# done
|
||||
#
|
||||
# Env overrides:
|
||||
# RUNNER_DIR — where to clone / find the runner (default: ../quic-interop-runner)
|
||||
# LOG_DIR — qlog / pcap output (default: $RUNNER_DIR/logs)
|
||||
@@ -145,7 +158,7 @@ fi
|
||||
# sim/server/client`, which Docker enforces globally regardless of
|
||||
# COMPOSE_PROJECT_NAME. Two simultaneous invocations collide on
|
||||
# `docker create container "sim"`. Run sequentially:
|
||||
# for peer in aioquic picoquic quic-go; do
|
||||
# for peer in aioquic picoquic quic-go quinn; do
|
||||
# quic/interop/run-matrix.sh -s $peer -t handshake,chacha20,...
|
||||
# done
|
||||
#
|
||||
|
||||
Reference in New Issue
Block a user