quinn is now treated as a flawless-required interop target alongside
aioquic, picoquic, and quic-go. Most Rust-based Nostr/MoQ relays our
users run their servers on are built on quinn, so an interop regression
there is a user-visible regression.
Validated by a 3-round flakiness sweep (1 full matrix + 2 audio-critical
subsets) — zero result flakiness across 528 test executions across all
four peers, with two environmental docker-compose stall classes documented
separately.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`LC_CTYPE=C tr -dc '[:alnum:]' </dev/urandom` in the upstream runner's
certs.sh trips "tr: Illegal byte sequence" on macOS — LC_CTYPE alone
doesn't override the runtime locale chain. Only the amplificationlimit
testcase exercises this loop (chain length > 1 → fakedns SAN inflation),
but if it aborts the runner stops the whole matrix BEFORE any later
test in TESTCASES_QUIC runs: handshakeloss, transferloss,
handshakecorruption, transfercorruption, ipv6, v2, rebind-port,
rebind-addr, connectionmigration, and the goodput/crosstraffic
measurements all silently never execute. The post-mortem summary
shows the 12 testcases that ran before the abort and looks deceptively
complete.
Add an idempotent `sed` step to run-matrix.sh that rewrites the line
to `LC_ALL=C tr` on every invocation, plus a plan-file note so the
next person hitting the deceptive summary doesn't repeat the
diagnosis. The patch is a working-tree edit to ../quic-interop-runner/,
not a fork; re-applied on every clone.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two paired changes:
(1) run-matrix.sh now tees the runner's full stdout (pre-grep-
filter) to <RUN_LOG_DIR>.stdout.log — sibling rather than
inside the log dir because run.py refuses to start if its own
--log-dir already exists.
(2) summarize-matrix.sh checks that file FIRST when searching for
'Test: X took Y, status:' lines — it has the authoritative
runner output that wasn't being saved before.
Old runs (without the tee) fall back to qlog inspection.
Single command for the diagnostic loop:
DEBUG=1 ./quic/interop/run-matrix.sh -s aioquic -t multiplexing
./quic/interop/inspect-multiplexing.sh
Previously users had to remember to 'cd quic/interop && make build
DEBUG=1' as a separate step, which is easy to forget.
The 2026-05-07 qlog post-streamsLock-fix shows the smoking gun:
- 1406 packets with exactly 1 stream frame each
- 1 packet (out of 1407) with >1 stream frames
- first stream packets at 1665ms / 1709ms (one RTT apart)
Wire shape says: writer is NOT bursting 64 streams per drain.
Hypothesis: connBudget exhaustion. Trace:
Iteration 1 of buildApplicationPacket:
streamA.takeChunk(maxBytes = min(streamCredit, connBudget))
→ returns 50-byte chunk
→ connBudget -= 50
Iteration N: connBudget == 0
streamN.takeChunk(maxBytes = 0)
→ returns null (fresh-bytes path: cap==0 ⇒ null)
→ skip
→ next iteration also skips
→ drain returns 1-stream packet
Wait for peer's MAX_DATA (one RTT)
→ connBudget bumps by maybe 50 bytes
→ emit one more stream
→ repeat
This matches the 40ms-per-stream cadence in the qlog exactly.
If the hypothesis is right, peer's initial_max_data is too small and
we're connection-flow-control bound by design (or by aioquic-qns
config). Three new sections in inspect-multiplexing.sh:
1. peer transport_parameters — directly shows initial_max_data
2. MAX_DATA arrivals — confirms the cadence + delta-per-bump
3. per-packet stream_id — confirms each packet carries a different
stream's first chunk
Also filtered the runner's "Generated random file" + "Requests:"
spam from run-matrix.sh output (separately requested).
Re-run inspect on the existing log dir to verify (no new matrix run
needed):
./quic/interop/inspect-multiplexing.sh
If initial_max_data is small, the fix is on us — we should pre-
advertise a larger initial_max_data on our side AND push for a
larger one from the peer (via setting our initial_max_data so peer
knows we can receive a lot, which may inform their MAX_DATA cadence).
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Two corrections from the user's just-completed run:
1. The runner does not have a 'versionnegotiation' testcase. Available
list output: handshake, transfer, longrtt, chacha20, multiplexing,
retry, resumption, zerortt, http3, blackhole, keyupdate, ecn,
amplificationlimit, handshakeloss, transferloss, handshakecorruption,
transfercorruption, ipv6, v2, rebind-port, rebind-addr,
connectionmigration. (`v2` exists but tests QUIC v2 protocol
support — we're v1-only, so it would correctly fail.)
The :quic-side VN code from agent A stays as defensive support for
any future server that throws a VN at us; just no testcase wires
it up.
2. run-matrix.sh is NOT safe to run concurrently. The runner's
docker-compose.yml hardcodes container_name: sim/server/client —
Docker enforces those globally regardless of COMPOSE_PROJECT_NAME.
Documented the limitation + the recommended sequential loop in
the script's comments.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
The runner aggregates container stderr verbatim, which gave us a
~50-line block of routing setup, NIC checksum offload toggles,
container lifecycle messages, and the long Command: WAITFORSERVER=...
line for every single testcase. Useful once for triage; pure noise
across a multi-test matrix run.
Two changes:
- run-matrix.sh pipes the runner output through a grep -Ev filter
that drops the boilerplate while keeping outcomes (Test: ... took /
status, the summary table, server's Starting server, sim scenario
+ capture lines, and any Python tracebacks). VERBOSE=1 bypasses
the filter for debugging.
- InteropClient drops its own pre-test header dump unless
QUIC_INTEROP_DEBUG=1 is set; the per-GET success line goes silent
too — failures still print as before.
Net result: a passing test reduces from ~50 noise lines to ~5
meaningful lines (test name, time, status, summary table).
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
pyshark's pcap reader calls asyncio.get_event_loop_policy().get_event_loop(),
which raises RuntimeError on Python 3.14 (deprecated in 3.12, removed in
3.14). Symptom: the matrix run completes the actual interop test
successfully but the runner's pcap-validation step crashes before it can
emit the result, dropping a 100+ line traceback.
run-matrix.sh now scans for python3.13 → 3.12 → 3.11 → python3, picking
the first that's both installed and < 3.14, and creates the venv with it.
Existing venvs created with 3.14 keep working (only matters for fresh
clones); for an existing broken venv, rm -rf .venv and re-run.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
run.py refuses to start if --log-dir already exists (interop.py:81-82
calls sys.exit). Previous script eagerly mkdir'd $LOG_DIR, which made
the second run always fail.
New layout: $LOG_DIR is the parent (created if missing), each
invocation writes to $LOG_DIR/run-YYYYmmdd-HHMMSS/. Preserves history
of past runs instead of overwriting; latest is `ls -t $LOG_DIR | head -1`.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Upstream quic-interop-runner split its config into implementations_quic.json
+ implementations_webtransport.json so QUIC and WebTransport endpoint
registrations don't collide. Schema is unchanged; just the filename.
Also updated the Makefile comment + plan doc for consistency.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
One-shot wrapper that clones quic-interop-runner alongside this repo if
missing, sets up its venv, merges our implementations.json snippet, builds
the endpoint image, then invokes run.py with passed-through args. Every
step is idempotent so repeated invocations just iterate.
Designed to script only the per-run loop, not first-time tooling install
(Docker Desktop, Homebrew, Wireshark prefs) — those are GUI / one-time
steps where a script either fails awkwardly or hides what the user is
consenting to.
Cross-platform install hints (apt-get on Linux, brew on macOS) on missing
prereqs. Daemon liveness check (docker info) catches the common macOS
"Docker Desktop installed but not running" trap. SKIP_BUILD=1 escape
hatch for tight image-unchanged inner loops.
Plan doc updated: run-matrix.sh is now the documented happy path; the
manual jq-merge sequence is kept as a fallback.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Vitor Pamplona