The previous coroutineScope { urls.map { async { ... } }.map { it.await() } }
pattern was vulnerable to a single hung stream blocking the whole await
chain — even though the others completed, sequential .await() iteration
would hang on the slowest forever. With multiplexing's hundreds of
streams, the probability of at least one having an issue (lost FIN,
slow consumer hitting channel-saturation thresholds, etc.) is high.
Wrap each get() in withTimeoutOrNull(PER_STREAM_TIMEOUT_SEC). A timed-out
stream surfaces as GetResponse(status=0); the caller's status != 200
check counts it as a failure but doesn't block the loop.
Doesn't fix throughput — that needs separate work on per-stream
backpressure and parser fairness. Just makes the failure mode visible
(status=0 → "failed file") rather than hidden (whole test times out
because of one bad stream).
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
aioquic multiplexing qlog showed:
t=31424: still receiving STREAM frames
t=31493: PTO timer expired
t=31507: connection_closed (owner: local)
We were still actively transferring when our 30s timeout fired. The
multiplexing testcase generates many small files (3431 sim packets
captured for the run) and download throughput on Mac+Rosetta is
dominated by per-write filesystem overhead in the Docker volume mount.
60s gives enough headroom without making fast-completing tests slower.
Doesn't fix any real protocol issue — just lets the test budget match
the workload.
If multiplexing still fails after this, the next investigation is the
sequential `.map { it.await() }` pattern in runTransferTest: a single
hung stream blocks the await chain even though others completed.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
aioquic still 0/7 after the close-race fix because there's a deeper
issue. QlogWriter.writeLineLocked() called writer.flush() on every
event. On macOS Docker Desktop the filesystem is virtualized and
per-event flush is multi-millisecond. The qlog hooks fire from
QuicConnectionWriter.drainOutbound() which runs INSIDE
conn.lock.withLock { ... } on the send loop. Every flush stalls
that lock; the receive loop blocks indefinitely trying to acquire
it for feedDatagram().
Symptom matched: client.sqlog showed our ClientHello packet_sent
at t=400ms, then NOTHING for the rest of the test — no
packet_received (server's response never decoded), no PTO probe
(send loop stuck behind disk IO).
Fix: remove per-event flush. close() flushes once at the end of
the run, which gives us the trace for everything except a hard
JVM kill mid-test (acceptable trade). BufferedWriter still buffers
in-memory; we don't lose events, just don't sync them to disk
synchronously.
Should restore aioquic to 5/7. picoquic was already 5/7 because
its server-response timing apparently let our send loop drain
between flushes; aioquic's faster turnaround tripped the lock more
reliably.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
aioquic full-matrix run came back 0/7 with stack traces:
java.io.IOException: Stream closed
at QlogWriter.writeLineLocked(QlogWriter.kt:289)
at QlogWriter.onPacketSent(QlogWriter.kt:126)
at QuicConnectionWriter.emitQlogSent(...)
The send loop runs concurrently with InteropClient's teardown sequence:
1. driver.close() — launches CLOSING → CLOSED async
2. qlogWriter?.close() — closes the file
3. delay(50)
4. scope.cancel() — kills coroutines
Between (2) and (4), the send loop is still alive and emits
packet_sent for the CONNECTION_CLOSE Initial. QlogWriter.writeLineLocked
threw IOException into the send loop, propagated up, took down the
connection — including connections that had completed transfer
successfully.
Fix: QlogWriter swallows post-close IOExceptions silently. An observer
must NOT break the connection it's observing — that's the whole NoOp
contract. Adds @Volatile closed flag, latches it both on explicit
close() and on the first IOException; subsequent emits short-circuit.
picoquic was 5/7 (predictable: M + S still open) so the regression was
specifically aioquic-timing-sensitive. With this fix aioquic should
return to 5/7 too.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Agent B's QlogWriter previously lived only in :quic's jvmTest scope,
hooked into the standalone InteropRunner. Brings it into :quic-interop
proper:
- QlogWriter.kt + QlogWriterTest.kt copied into :quic-interop with
package com.vitorpamplona.quic.interop.runner.
- Jackson dep added to :quic-interop's build.gradle.kts.
- InteropClient reads $QLOGDIR (the runner sets it inside the
container per docker-compose.yml). When set, opens a QlogWriter
at <QLOGDIR>/client.sqlog, passes as QuicConnection.qlogObserver,
closes on every exit path (handshake_failed / udp_failed /
transfer_timeout / ok).
- QUIC_INTEROP_DEBUG=1 now prints qlogdir alongside the other env
fields.
Net effect: any future runner-driven test failure dumps a structured
qlog file the user can drag straight into qvis (qvis.quictools.info)
to see frame-by-frame what the client decided to do. The retry test
failure on aioquic + picoquic that we still need to debug now produces
a usable artifact.
NOTE: the QlogWriter copy in :quic's jvmTest stays in place as the
helper for the standalone InteropRunner main(). Slight duplication;
acceptable while :quic-interop is its own module.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Two corrections from the user's just-completed run:
1. The runner does not have a 'versionnegotiation' testcase. Available
list output: handshake, transfer, longrtt, chacha20, multiplexing,
retry, resumption, zerortt, http3, blackhole, keyupdate, ecn,
amplificationlimit, handshakeloss, transferloss, handshakecorruption,
transfercorruption, ipv6, v2, rebind-port, rebind-addr,
connectionmigration. (`v2` exists but tests QUIC v2 protocol
support — we're v1-only, so it would correctly fail.)
The :quic-side VN code from agent A stays as defensive support for
any future server that throws a VN at us; just no testcase wires
it up.
2. run-matrix.sh is NOT safe to run concurrently. The runner's
docker-compose.yml hardcodes container_name: sim/server/client —
Docker enforces those globally regardless of COMPOSE_PROJECT_NAME.
Documented the limitation + the recommended sequential loop in
the script's comments.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Two integrations from overnight agents:
1. agent C — peer-uni-stream drainer for the H3 multiplexing fix.
Http3GetClient.init() now takes a CoroutineScope and calls
conn.drainPeerInitiatedUniStreamsIntoBlackHole(scope) to consume +
discard the server's three uni streams (control, qpack-encoder,
qpack-decoder). RFC 9114 §6.2 mandates the client read these;
pre-fix their bytes accumulated in 64-chunk per-stream channels
until parser overflow tore down the connection with INTERNAL_ERROR
~4.5s into a multiplexing run.
2. agent A — versionnegotiation testcase wired into dispatch.
Adds the testcase to the runTransferTest route and threads
QuicVersion.FORCE_VERSION_NEGOTIATION as the initial version when
testcase=='versionnegotiation'. The QuicConnection's RFC 9000 §6
VN flow (applyVersionNegotiation) takes over from there, retrying
with v1 once the server replies with its supported list.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
The previous commit (e5bbf8509) switched ALPN per testcase based on
quic-interop-runner convention (h3 for http3/multiplexing, hq-interop
otherwise). That broke picoquic, which had been 4/4 green: picoquic-qns
strictly registers h3 ALPN and rejects hq-interop. Different servers
disagree on which ALPN they configure for the same testcase:
- quic-go-qns — strictly hq-interop for non-http3
- aioquic-qns — accepts either
- picoquic-qns — strictly h3 for all testcases
There's no per-testcase convention all peers honor. Right move is the
TLS-spec-supported one: offer BOTH h3 and hq-interop in the ClientHello,
let the server pick whichever matches its config, then dispatch the GET
client by `tls.negotiatedAlpn` after the handshake completes. http3 and
multiplexing still restrict to h3 only since they exercise H3 framing.
Brings picoquic back to fully green and should unblock quic-go for the
non-http3 testcases.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Two more testcases now dispatched through runTransferTest:
- retry — exercises the RFC 9000 §17.2.5 + RFC 9001 §5.8 Retry
handling that landed in d03e17981 / 671f9c705 (DCID swap,
integrity-tag verify, key re-derivation, token threading).
- ipv6 — same flow over an IPv6 socket. JDK's DatagramChannel.connect
handles the v6 address resolution natively; if anything
breaks it'll be an actual bug worth surfacing.
Plan doc updated to reflect Phase 3 landings (ALPN per testcase,
HqInteropGetClient, multi-stream FIN delivery fix) and the current
validation matrix across aioquic / picoquic / quic-go.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
quic-go-qns interop revealed two coupled gaps in our endpoint:
1. ALPN per testcase. The runner convention (followed strictly by
quic-go-qns, lazily by aioquic / picoquic) is:
- testcase 'http3' / 'multiplexing' → ALPN 'h3' (full HTTP/3)
- everything else → ALPN 'hq-interop'
(HTTP/0.9 over QUIC)
We had hardcoded 'h3'. quic-go closed every non-http3 connection
with CRYPTO_ERROR 0x178 (TLS no_application_protocol).
2. We had no HQ-interop client. HQ is dead simple: open bidi, send
`GET /path\r\n` raw, FIN, read body verbatim until server FINs.
No framing, no QPACK, no control stream.
This commit adds:
- GetClient interface + GetResponse data class extracted from
Http3GetClient so the runner code dispatches uniformly.
- HqInteropGetClient — the 30-line HQ-interop GET implementation.
- Alpn enum (H3, HQ_INTEROP) wired through main() → runTransferTest
→ QuicConnection.alpnList. Picked from the testcase name per the
convention above.
Validated locally: :quic-interop:test green; :quic-interop:installDist
clean. Will need a real run against quic-go to confirm the fix.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
The runner aggregates container stderr verbatim, which gave us a
~50-line block of routing setup, NIC checksum offload toggles,
container lifecycle messages, and the long Command: WAITFORSERVER=...
line for every single testcase. Useful once for triage; pure noise
across a multi-test matrix run.
Two changes:
- run-matrix.sh pipes the runner output through a grep -Ev filter
that drops the boilerplate while keeping outcomes (Test: ... took /
status, the summary table, server's Starting server, sim scenario
+ capture lines, and any Python tracebacks). VERBOSE=1 bypasses
the filter for debugging.
- InteropClient drops its own pre-test header dump unless
QUIC_INTEROP_DEBUG=1 is set; the per-GET success line goes silent
too — failures still print as before.
Net result: a passing test reduces from ~50 noise lines to ~5
meaningful lines (test name, time, status, summary table).
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Two coupled bugs in our endpoint that the runner just surfaced:
1. The runner mounts \$CLIENT_DOWNLOADS to /downloads as a Docker volume
(per quic-interop-runner's docker-compose.yml `client.volumes`).
There is no DOWNLOADS env var. Our endpoint was reading \$DOWNLOADS,
getting null, and (for handshake/chacha20) skipping any download path.
Hard-code /downloads.
2. The handshake / chacha20 / handshakeloss testcases don't just verify
the handshake completes — they also require the requested file at
/downloads/<basename>. The runner's _check_files validator
reported "Missing files: ['intense-tremendous-firefighter']" while
our client side reported `handshake ok`. Route handshake-flavor
testcases through runTransferTest so the full H3 GET pipeline runs.
`chacha20` keeps the cipher-suite override (ChaCha20-only ClientHello);
`handshakeloss` reuses the same H3 flow against a lossy sim.
Also drops the now-dead runHandshakeTest + parseFirstTarget helpers.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
The quic-interop-runner exposes several testcases that drive the same
client logic but vary the network conditions injected by the ns-3 sim.
These don't need new client code on our side — they exercise the
existing handshake / transfer paths under loss / corruption / high-RTT /
cross-traffic, which is exactly the bug-finding signal we want.
Aliases added:
- transferloss, transfercorruption, longrtt, goodput, crosstraffic
→ transfer (H3 GET against varying sim configs)
- handshakeloss → handshake
Plan doc now lists every standard testcase and either marks it landed,
aliased, or explicitly unsupported with a written reason — so anyone
returning to this knows what's left and why each gap exists. Unsupported
set covers: versionnegotiation (writer hard-codes V1), resumption /
zerortt (no session ticket / 0-RTT), keyupdate (no KEY_PHASE handling),
retry (parser exists but not wired to feed-loop), rebinding-* (no client
migration), amplificationlimit (server-side), blackhole (inverse test),
ipv6 (UdpSocket v6 path unverified).
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Adds a minimal Http3GetClient (in :quic-interop, NOT :quic — interop-test
surface, not a production HTTP/3 client) that opens the three required
client uni streams (control + QPACK encoder + QPACK decoder per RFC 9114
§6.2.1), sends an empty SETTINGS, and per request opens a bidi stream,
encodes the four pseudo-headers via the existing literal-only QpackEncoder,
FINs, and reassembles HEADERS+DATA frames from the response.
Wires three testcases:
- transfer: GET each REQUESTS URL sequentially, write body to
\$DOWNLOADS/<basename>. status != 200 fails.
- http3: identical to transfer.
- multiplexing: same fetches but issued in parallel via
coroutineScope { async { … } } so request streams genuinely
overlap on the wire (what tshark verifies).
Out-of-scope (deliberate): GOAWAY, PUSH_PROMISE, dynamic QPACK table,
trailers, priority — none are required by these testcases.
Unit-tests round-trip the request encoding through the existing
Http3FrameReader + QpackDecoder so the wire format is verified without
needing a real peer.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Refactors SslKeyLogger to take the ClientHello random at flush() time
(removes the lookup-lambda + bindConnection ceremony) so the formatter
is testable without spinning up a real QuicConnection. Adds three
unit tests covering: the four NSS Key Log lines emitted in the right
order, flush idempotency, and lower-case hex encoding.
Also drops a checked-in implementations.json snippet
(quic-interop-runner-snippet.json) so a sibling clone of the runner
can be registered with one jq merge instead of hand-edited JSON.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Adds two debugging hooks to :quic so the interop endpoint can produce
artifacts that make the runner actually useful for finding bugs:
- TlsClient.clientRandom: capture and expose the 32-byte ClientHello
random so a SSLKEYLOG line can correlate to this connection.
- QuicConnection.extraSecretsListener: optional chained secrets
listener (default null, no-op for production callers). Fires
alongside the connection's own key-installation listener at every
encryption-level transition.
- QuicConnection.cipherSuites: knob to override the offered TLS
cipher suites in ClientHello.
InteropClient now:
- Writes NSS Key Log Format lines to $SSLKEYLOGFILE when set, so
Wireshark can decrypt the sim's pcap captures.
- Implements the `chacha20` testcase by offering only
TLS_CHACHA20_POLY1305_SHA256 — exercises the ChaCha20 AEAD path
end-to-end against a peer.
Defers QLOGDIR (needs qlog observer infrastructure across packet/
frame/recovery layers — own design doc) and `versionnegotiation`
(needs the writer to accept a configurable initial QUIC version).
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Adds :quic-interop, a JVM-only module at quic/interop/ that implements
the quic-interop-runner Docker contract so we can drive matrix tests
against aioquic / quiche / picoquic / msquic / ngtcp2 / etc. as a
bug-finding harness.
Phase 0 supports only the `handshake` testcase; everything else returns
127 so the runner skips rather than fails. SSLKEYLOGFILE / QLOGDIR
plumbing is deferred to Phase 1.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Claude