The qlog confirmed the writer emits 1 STREAM frame per packet on the
live wire, while MultiplexingCoalescingTest + MultiplexingAioquicTpsTest
both show ~9 streams per packet under synchronous drain. So the bug is
in the live driver flow — concurrent send loop + parser feed +
real-socket interleaving — and not in buildApplicationPacket itself.
To localize: add an opt-in trace at the END of buildApplicationPacket
that dumps per-drain state when QUIC_INTEROP_DEBUG=1:
[writer.app] frames=N stream_frames=K streamsView=M active=A
packetBudget_remaining=R connBudget_initial=C
- frames vs stream_frames tells us if non-stream frames (ACK,
MAX_DATA, MAX_STREAM_DATA) are bloating the packet
- active vs streamsView tells us if isClosed filter dropped streams
- packetBudget_remaining tells us if we hit the 64-byte break early
- connBudget_initial tells us if conn flow control was zero
Wired three pieces:
1. WriterDebug.kt — a single @Volatile boolean owned by commonMain,
`writerDebugEnabled`. Off by default.
2. InteropClient.main flips it to true if QUIC_INTEROP_DEBUG=1 is set
in the env.
3. Dockerfile + Makefile accept --build-arg DEBUG=1 (or `make build
DEBUG=1`) to bake the env var into the image.
Usage:
cd quic/interop
make build DEBUG=1
cd ../..
./quic/interop/run-matrix.sh -s aioquic -t multiplexing
cat ../quic-interop-runner/logs/run-*/aioquic_amethyst/multiplexing/output.txt | grep '^\[writer'
When off, cost is one volatile read in the writer hot path — negligible.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Two adds for the multiplex investigation.
(1) inspect-multiplexing.sh: previous histograms aggregate over the
whole run. Add full-frame-array dump of the first 10 packet_sent
AND first 10 packet_received events so we can see whether the
FIRST chunk burst (~13 streams in one packet) or dribbled (1 per).
(2) MultiplexingAioquicTpsTest: synchronous drain test using EXACTLY
the TPs aioquic gave us in the failing run (initial_max_data=1MB,
initial_max_stream_data_bidi_remote=1MB, initial_max_streams_bidi=
128) and ~80-byte HEADERS-frame-sized payloads. PASSES with 7
packets / 9.1 streams per packet — proving the writer's coalescing
is fine under aioquic's flow-control budget. So the bug is NOT in
the writer; it's in the live driver flow that
MultiplexingCoalescingTest doesn't exercise (concurrent send loop
+ parser + real socket).
The first-10 dump from inspect should localize this further:
- if first packet has 13 stream frames → writer burst, bug is
server-side timing or driver-loop scheduling
- if first packet has 1 stream frame → writer producing 1 per call
in production for some condition my synchronous test doesn't
cover
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Round-2 audit of the openBidiStreamsBatch / openUniStreamsBatch landing.
(1) Tests overpromised. The previous "holds streamsLock for the whole
batch" test only verified the API didn't crash and stream ids were
unique. A future regression that released the lock between opens
(the 2026-05-06 bug shape) would not break it.
Added `assertTrue(client.streamsLock.isLocked)` inside both batch
init lambdas. Now the test name matches what's verified.
(2) `*Batch` docstrings didn't warn that `init` runs under
streamsLock. A naive caller might do encoding / IO inside, defeating
the lock-hold-time goal that motivated pre-encoding outside. Added
the warning + the canonical caller shape (encode outside, enqueue
inside) to both function docs.
(3) Empty-batch corner: an empty `items` list was still acquiring the
lock and entering withLock. Added an `if (items.isEmpty()) return
emptyList()` short-circuit. New test pins the contract — `init`
must not run for an empty batch.
Test count: 6 → 7. All green; no production-API change.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Audit follow-up. Two cleanups consolidated into one commit since they
share the goal of "make the lock contract obvious from the API".
(1) Remove LevelState.levelLock entirely.
The lock-split refactor introduced a per-level Mutex with a docstring
claiming the writer/parser would acquire it around encode + sentPackets
record + ACK observation. Neither actually does. SendBuffer's internal
synchronized(this) is what serializes cryptoSend mutations against
takeChunk and markAcked. handlePtoFired's levelLock acquisition was
the only production usage and it serialized only against itself.
Removed:
- LevelState.levelLock
- handlePtoFired's withLock wrapper (now a non-suspend fun)
- All docstring references to a third lock domain
The pre-existing sentPackets HashMap race (writer mutates under
streamsLock, parser reads without sync) is unchanged — out of scope
for this PR. Acquisition order is now `lifecycleLock → streamsLock`,
flat.
(2) Add openBidiStreamsBatch + openUniStreamsBatch — the bug-resistant
high-level API for the prepareRequests / moq audio-rooms patterns.
The previous shape required callers to manually do
`streamsLock.withLock { repeat(N) { openBidiStreamLocked() ... } }`.
That contract regressed twice on this very branch: once held the wrong
lock (lifecycleLock alias), once skipped the wrap entirely. Both shapes
silently emitted one STREAM per packet under multiplex load.
The new API encapsulates the lock + the per-item init lambda:
conn.openBidiStreamsBatch(items) { stream, item ->
stream.send.enqueue(encode(item))
stream.send.finish()
Handle(stream)
}
Callers physically cannot hold the wrong lock. Migrated:
- Http3GetClient.prepareRequests
- HqInteropGetClient.prepareRequests
Also added `openUniStreamLocked` + `openUniStreamsBatch` symmetric to
the bidi versions. moq audio-rooms eventually wants to open many uni
streams in burst; the same one-stream-per-packet bug lurks if every
open serializes through its own lock acquisition.
`openBidiStreamLocked` / `openUniStreamLocked` remain public (with
their `check(streamsLock.isLocked)` guards) for the rare custom-batch
callers that need to mix bidi+uni opens under a single hold. Most
callers should use the *Batch variants going forward.
Test coverage extended:
- openBidiStreamsBatch happy path
- openUniStreamLocked throws without streamsLock
- openUniStreamsBatch happy path
Six tests in BatchedOpenLockContractTest now pin the contract.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Audit of recent multiplex / PTO commits surfaced three concrete
improvements:
1. **Pre-encode requests outside streamsLock** in both prepareRequests
impls. QPACK encoding (Http3GetClient) and string formatting
(HqInteropGetClient) are non-trivial under multiplex load — 1999
paths means we were holding streamsLock across all 32 chunks ×
~2KB of QPACK encoding per chunk = ~64 KB of CPU work serialised
against the send loop. Now done outside the lock.
2. **Fix O(N²) byte merging in MultiplexingRoundTripTest.** Previous
shape allocated a new array per STREAM frame; for real-size
payloads this would dominate test runtime. Switched to a per-
stream MutableList<ByteArray> joined once at the end.
3. **Pin coalescing end-to-end in MultiplexingRoundTripTest.**
Pre-existing test verified per-stream content arrived but said
nothing about the wire shape. Added totalDatagrams ≤ 12 assertion
so the matrix's "one stream per datagram" failure mode would
break the test instead of passing silently.
Audit also surfaced two non-actionable items, flagged for future
cleanup but not changed in this commit:
- LevelState.levelLock docstring claims writer/parser acquire it
around sentPackets mutations; in practice neither does. The
handlePtoFired call that takes levelLock currently serialises
only against itself; SendBuffer's internal synchronized is what
prevents the actual cryptoSend race. Kept the call (matches the
documented design intent and future-proofs against the writer
actually taking levelLock) but the docstring is stale.
- openBidiStreamLocked's check(streamsLock.isLocked) catches "no
lock" and "wrong lock" callers but not "another coroutine holds
streamsLock and I'm calling without holding it" — kotlinx Mutex
doesn't expose owner-aware checks without an explicit owner arg
we don't pass. Acceptable since the bug we just fixed and any
future regression in the same shape are caught.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
aioquic interop multiplexing 2026-05-06 qlog post-mortem:
- 2898 packets sent in 60s, each carrying ONE STREAM frame
- server log: streams created/discarded strictly serially, ~30-40ms apart
- 1421/2000 files completed before the runner's 60s timeout
- shape ≈ 1 RTT per stream — wire was emitting one stream per datagram
Cause: Http3GetClient.prepareRequests + HqInteropGetClient.prepareRequests
both did `conn.lock.withLock { ... openBidiStreamLocked() }`. Post the
lock-split refactor `conn.lock` is the deprecated alias for lifecycleLock.
The writer's drainOutbound takes streamsLock — not lifecycleLock — so the
send loop interleaved between every two openBidiStreamLocked calls,
draining one stream's data per pass.
Fix:
1. Both prepareRequests impls now use conn.streamsLock.withLock.
2. openBidiStreamLocked now `check`s streamsLock.isLocked at entry
so this can never silently regress again — calling it without the
lock (or with the wrong lock) throws IllegalStateException with
a message naming streamsLock as the lock to acquire.
3. New BatchedOpenLockContractTest pins the contract:
- calling openBidiStreamLocked WITHOUT any lock throws
- calling openBidiStreamLocked while holding lifecycleLock throws
(the exact regression shape)
- calling openBidiStreamLocked while holding streamsLock works
(happy path)
The runtime check is the regression-proof part: future callers physically
cannot hold the wrong lock without the test (and prod) blowing up at the
first call site.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
The runner's `multiplexing` testcase opens N parallel bidi streams,
each downloading one file, and asserts every file's content lands
on the right stream. Existing tests cover pieces of that:
- MultiplexingThroughputTest: opens 1000 streams in <2s — measures
lock contention but never moves bytes server-side.
- MultiplexingCoalescingTest: pins that 64 streams coalesce into ≤6
packets — encoder contract, no end-to-end.
- MultiStreamFinDeliveryTest: server pushes responses to 50 streams,
client surfaces every FIN — but the CLIENT never sends a STREAM
frame in that test, so any regression in the writer's request-
side multiplex path is invisible.
This test runs the full request → response loop:
1. Open 64 parallel bidi streams
2. Each enqueues a tiny request + FIN
3. Drain client outbound → decrypt → assert all 64 STREAM frames
made it across, with their request bytes intact
4. Server sends one response per stream + FIN
5. Per-stream incoming.toList() must yield the expected response
Failures call out the specific stream id, so a regression points
at "stream X dropped its FIN" instead of a generic timeout.
64 streams keeps wall-clock under a second; the bug class the test
guards against (per-stream loss / mis-routing) fires identically at
64 and 1999.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
The existing PtoCryptoRetransmitTest simulated the driver inline:
it set pendingPing=true and called requeueAllInflightCrypto by hand,
then asserted the next drain emitted CRYPTO. That checked the helpers
worked but never noticed when the DRIVER stopped calling
requeueAllInflightCrypto — which is exactly the regression that bit
us in commits c0d7b6031 (qlog merge) and again in cf2303a38
(lock-split refactor).
Extract the PTO-fired logic from QuicConnectionDriver.sendLoop into
a top-level internal helper handlePtoFired(conn). Driver and test now
both call the same function — if anyone unwires the requeue from
that helper or rewrites sendLoop to do volatile-only updates, the
test breaks.
Verified the regression-catch by stripping the requeue from
handlePtoFired and re-running the test: it fails with an
AssertionError on the PTO retransmit packet check, as expected.
Restored the helper, test green again.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
aioquic interop multiplexing qlog (the smoking gun):
packet_sent PN=0 Initial frames=[crypto] (ClientHello)
packet_sent PN=1 Initial frames=[ping] (PTO probe — bare PING)
packet_received connection_close 0x0
"Packet contains no CRYPTO frame"
This is the SAME bug commit c0d7b6031 fixed; the lock-split refactor
(ef4bb9998) re-wrote the driver's PTO branch to use streamsLock and
inlined the @Volatile pendingPing/consecutivePtoCount writes — but
dropped the requeueAllInflightCrypto call that c0d7b6031 had wired
under the (now-renamed) connection.lock.
Restored under levelLock for the appropriate level. SendBuffer's
internal synchronized covers correctness, but the level lock keeps
us from racing the writer's takeChunk mid-build.
The writer's comment at QuicConnectionWriter.kt:421-431 already
documented this contract — it was just unwired on the driver side.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Tier 1 lock-split agent's worktree was based on main and didn't carry
the Retry-handling work (fields retryToken / retryConsumed, applyRetry
method's references to them). Merge with -X theirs nuked those.
Restored:
- retryToken + retryConsumed @Volatile fields on QuicConnection,
re-attached to applyRetry in the lock-split-merged file.
- LevelState.resetForVersionNegotiation now uses pnSpace.resetForRetry()
to reset the PN counter in place — pnSpace is `val` post-refactor,
direct re-assignment doesn't compile. The naming is historical;
the underlying semantics (zero PN counter + clear received side)
are correct for both VN and Retry-with-fresh-PN cases.
- openBidiStream split into the public suspend wrapper +
openBidiStreamLocked() (caller holds streamsLock). Restores the
batched prepareRequests path's ability to open N streams under one
lock hold.
- close() — local var firedQlog hoisted out of withLock block.
Test suite runs clean. Three deprecation warnings remain on
PtoCryptoRetransmitTest + QlogObserverTest still using the
backward-compat conn.lock alias; left for follow-up cleanup.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
The single connection-wide `QuicConnection.lock` mutex serialised every
critical path: the read loop's `feedDatagram`, the send loop's
`drainOutbound`, and every public mutator (`openBidiStream`,
`streamById`, `flowControlSnapshot`, ...). The multiplexing testcase
opens hundreds of bidi streams in parallel and was capped at ~25
streams/sec by lock contention against the I/O loops.
Phase 1 of the lock split (see
`quic/plans/2026-05-08-lock-split-design.md`) introduces three
domain-specific mutexes:
- `streamsLock` — streams registry, datagram queues, stream-id
counters, connection-level flow-control bookkeeping, pending-
retransmit maps for control frames
- `LevelState.levelLock` (one per encryption level) — per-level
pnSpace / sentPackets / ackTracker / CRYPTO buffers
- `lifecycleLock` — status transitions, close reason/error code
Acquisition order: `lifecycleLock < streamsLock < levelLock`.
Per-stream `synchronized(this)` blocks inside SendBuffer/ReceiveBuffer
remain at the leaf — never acquire any QuicConnection mutex while
holding a per-stream lock.
The legacy `lock: Mutex` field is preserved as a deprecated alias of
`lifecycleLock` for source-compatibility with external test harnesses;
new code MUST use the appropriate domain lock.
Highlights:
- `feedDatagram` / `drainOutbound` now require the caller to hold
`streamsLock`; the driver wraps each call. Phase 1 keeps the whole
feed/drain inside `streamsLock` for safety; phase 2 (deferred) will
split frame-collection from encrypt + sentPackets-record so app
coroutines can intersperse during the encrypt window.
- `pendingPing`, `peerTransportParameters`, `status`,
`handshakeComplete` are now @Volatile so observers read them
without a lock.
- `markClosedExternally` no longer needs any lock (status is
@Volatile, signals are channel-thread-safe).
- Driver's PTO bookkeeping uses the volatile fields directly — no
lock needed.
- Tests that manually acquired `conn.lock` to call
`getOrCreatePeerStreamLocked` / `onTokensAcked` / `onTokensLost`
now acquire `streamsLock` (the domain those routines mutate).
- New `MultiplexingThroughputTest` locks in the contract: 1000
parallel `openBidiStream` calls must complete in <2 s.
Test plan:
- `:quic:jvmTest` — 294 tests pass (293 prior + 1 new throughput).
- `MultiplexingThroughputTest`: 1000 bidi streams in 52 ms
(~19,000 streams/sec on the in-memory pipe), well above the
250+/sec target.
- `:nestsClient:compileKotlinJvm` — clean, no API breaks.
- `./gradlew :quic:spotlessApply` — clean.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Diagnosis from yet another qlog round: streams/packet still ~1 even
with the prepareRequest/awaitResponse split. Root cause: openBidiStream
is suspend due to lock.withLock, and each call releases the lock between
iterations. The send loop is queued on the lock; it grabs it the moment
we release, drains the one stream of data we just enqueued, and the
next prepareRequest call has to re-acquire after the send loop releases.
Net: one stream per drain per packet, same useless coalescing as before.
Fix is structural:
- QuicConnection.openBidiStreamLocked() — public, lock-not-acquired
version of openBidiStream. Caller MUST hold conn.lock.
- GetClient.prepareRequests(authority, paths) — batch API that
holds conn.lock once, opens + enqueues all N streams in a single
critical section, releases. Send loop can't interject; when it
next drains it sees ALL N streams' data ready and packs them
into coalesced packets.
- Http3GetClient + HqInteropGetClient: implement prepareRequests
using openBidiStreamLocked under conn.lock.withLock { ... }.
- InteropClient's chunked-multiplex loop: uses prepareRequests
(batch) instead of N x prepareRequest.
Single-stream paths still use prepareRequest / get(); behavior unchanged.
The fundamental architectural improvement (per-stream / per-level lock
split, or actor-model dispatch) is a follow-up; this commit gets us
the throughput we need from the existing single-mutex shape by holding
the lock for the full chunk's worth of work.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Two unit tests for the multiplexing throughput problem we just fixed
in the InteropClient (commit bc19e90c1):
1. `64 streams enqueued before drain coalesce into a small fixed
number of packets` — opens 64 bidi streams, enqueues 50 bytes +
FIN on each (no drain between), drains everything, asserts
≤6 packets and ≥10 streams/packet. Pre-fix shape (each enqueue
followed by an immediate single-stream drain) would emit 64 packets.
2. `1000 streams enqueued in batches of 64 produce a tractable packet
count` — stress version mirroring the runner's 1999-file
multiplexing test. Asserts ≤150 packets total for 1000 streams.
Both tests run in <300ms on the in-memory pipe — fast iteration for
debugging the multiplexing-throughput regression cycle without
spinning up Docker.
These pin the contract that ZERO drains between enqueues = packets
batch many streams' frames. The runner-side fix (split GetClient.get
into prepareRequest + awaitResponse, batch prepareRequests serially,
wake once) ensures we hit this shape in real interop.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Multiplexing-throughput investigation (qlog against aioquic):
~25 streams/sec with 1453 GETs in 58s. The bottleneck under high
stream count was drainOutbound's per-call O(N log N) sort over the
ENTIRE stream list (including streams that have already FIN'd both
ways and have nothing to send).
Two cheap optimizations to drainOutbound's stream iteration:
1. Filter to !isClosed streams BEFORE sort. Most streams under
bursty multiplexing loads are done; iterating them is wasted.
2. Skip sortedByDescending entirely when every stream is at
default priority (priority == 0). The pre-priority round-robin
shape (insertion order) is preserved, satisfying the moq-lite
newer-sequence-stream priority contract by happenstance for
uniform-priority loads.
Drops drainOutbound's per-call cost from O(N log N) where N = total
streams to roughly O(active) under realistic loads. Multiplexing's
~2000 streams accumulated over a run drop down to maybe 64 active at
any moment (the chunk in flight).
Doesn't affect the moq-lite audio path's behavior (small N, default
priorities → both paths reduce to the same round-robin walk).
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
aioquic's retry test result, surfaced via qlog:
Check of downloaded files succeeded.
Client reset the packet number. Check failed for PN 0
Our applyRetry called LevelState.resetForVersionNegotiation, which
creates a fresh PacketNumberSpaceState() — resetting PN to 0. The
qlog confirmed: PN=0 sent at t=388 (pre-Retry ClientHello), then PN=0
again at t=1468 (post-Retry retried ClientHello). Same PN reused
across the boundary.
RFC 9001 §5.7 + RFC 9000 §17.2.5: the Initial PN namespace CONTINUES
across the Retry boundary. The new Initial keys are derived from the
new DCID, but PN doesn't reset. Reusing a PN under different keys
makes the runner's pcap-decryption check fail (it's also a security
concern in the general case, hence the strict spec rule).
Fix: new LevelState.resetForRetry that's identical to
resetForVersionNegotiation EXCEPT it preserves pnSpace. applyRetry
calls resetForRetry. Two regression tests updated to assert the
post-Retry Initial uses PN=1 (continues from PN=0 of the pre-Retry
attempt) rather than PN=0 (the buggy reset behavior).
For Version Negotiation the original semantics still apply (RFC 9000
§6.2: client treats VN as if the original Initial was never sent;
PN reset to 0 is correct).
This should bring the retry testcase from ✕(S) to ✓(S) against
servers that exercise the Retry path. The handshake / transfer
already succeeded over the Retry per the qlog (the server's check
"Check of downloaded files succeeded." passed); only the PN-reuse
flag was failing the test.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Direct evidence from aioquic interop qlog:
packet_sent PN=0 frames=[crypto] (ClientHello)
packet_sent PN=1 frames=[ping] (PTO probe — bare PING)
packet_received frames=[connection_close]
reason="Packet contains no CRYPTO frame"
aioquic strictly rejects pre-handshake Initials that contain no
CRYPTO frame. Our PTO probe was a bare PING, not a CRYPTO retransmit.
Agent 2's c9e036f72 implemented the right behavior: on PTO, the
driver requeues all inflight CRYPTO at the highest pre-application
level, so the next drain emits a fresh CRYPTO frame at the original
offset. The pendingPing flag stays as a fallback only used when no
CRYPTO is available to retransmit. That logic was wiped during the
qlog observer merge — driver's PTO branch only set pendingPing,
nothing requeued anything, probe was always bare PING.
Restored. Should fix the post-flush-fix regression where aioquic
went 1/7 (only transfer green) — once a packet was dropped or PTO
fired, the next probe was bare PING, aioquic closed with
"no CRYPTO frame", the test failed. Sequential tests on the same
matrix invocation likely intermittently passed/failed depending on
whether PTO fired before the response arrived.
The :quic helpers requeueAllInflightCrypto + highestPreApplicationLevel
already exist on the branch — just the driver call wasn't wired.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Same merge-from-main shape as the prior agent A integration: the qlog
agent's worktree didn't carry the version-negotiation work (QuicVersion
import, currentVersion / vnConsumed fields, applyVersionNegotiation),
the Retry work (extraSecretsListener / cipherSuites / applyRetry), or
the version-negotiation testcase wiring. Merge with -X theirs took the
qlog version of QuicConnection.kt + QuicConnectionWriter.kt + the
existing InteropRunner.kt wholesale, dropping those.
Restored:
- QuicVersion import in QuicConnection.kt + QuicConnectionWriter.kt +
the test-side InteropRunner.kt (also touched by agent B's qlog
hooks).
- extraSecretsListener / cipherSuites / initialVersion ctor params
on QuicConnection (qlogObserver kept; new qlog work landed).
Net result: all three overnight agents (A versionnegotiation, B qlog
observer, C peer-uni-stream drainer) now coexist on the branch with no
references missing. Full :quic:jvmTest green; :quic-interop:test +
installDist green.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
The agent A worktree was based on main, so its QuicConnection.kt
didn't carry the Retry handling from d03e17981. Merging with
-X theirs replaced the file wholesale, dropping applyRetry +
extraSecretsListener / cipherSuites constructor params + the
RetryPacket import in the parser.
Restored:
- extraSecretsListener + cipherSuites ctor params (used in tlsListener
+ the TlsClient construction site).
- applyRetry method, now using the version-negotiation-introduced
LevelState.resetForVersionNegotiation helper (functionally
equivalent to the prior restoreFromRetry it replaced).
- RetryPacket import in QuicConnectionParser.
Also dedupes a duplicate `originalClientHello` field that the merge
left both copies of (one from agent 3's Retry work, one from agent A's
VN work). Single field now serves both reset paths.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Variant (B) from the three-way fix menu in the multiplexing-interop
investigation: keep `:quic` strict about per-stream backpressure (the
audit-4 #3 "INTERNAL_ERROR: stream … consumer overflowed" tear-down
stays the contract for app-data overflow on bidi streams) but expose
an explicit, opt-in helper for peer-initiated UNI streams that the
application has decided it does not need to interpret.
Root cause confirmed in QuicConnectionParser.kt:290: when the server
opens its three RFC 9114 §6.2.1 peer-uni streams (CONTROL +
QPACK_ENCODER + QPACK_DECODER) and the H3 client does not consume
them, the parser routes their bytes into each stream's bounded
incomingChannel (capacity 64). Once the QPACK encoder issues
dynamic-table inserts beyond 64 chunks the next chunk overflows
trySend, sets QuicStream.overflowed, and the parser maps that to
markClosedExternally — the entire connection dies.
Notes on scope:
- The `Http3GetClient` and `:quic-interop` runner mentioned in the
investigation prompt do NOT exist on the `main` worktree this
branch starts from. The fix here is therefore `:quic`-only: the
public `awaitIncomingPeerStream` API was already sufficient for
an integrator to write the accept loop themselves; this commit
wraps the common case in `drainPeerInitiatedUniStreamsIntoBlackHole`
and updates the doc on `awaitIncomingPeerStream` so the next
integrator landing the H3 GET client doesn't hit the same trap.
- Variant (C) — silent default drain in `:quic` itself — was
deliberately rejected: defaults that swallow application bytes
are the misconfiguration we want type-system-or-API-explicit.
The new helper requires the caller to pass a CoroutineScope, so
opt-in is unmistakable in any callsite.
Regression test coverage in PeerUniStreamDrainTest:
- pre_fix_no_consumer_overflows_and_tears_down_connection — pushes
65 chunks (capacity + 1) on a SERVER_UNI stream with no consumer;
asserts the connection transitions to CLOSED. Pins the existing
backpressure contract.
- drainPeerInitiatedUniStreamsIntoBlackHole_keeps_connection_alive
— same setup but with the new helper running on a side scope;
pushes 256 chunks (4× capacity) and asserts the connection stays
CONNECTED. With the helper sabotaged, this test fails at
line 119 with status=CLOSED, confirming it actually exercises
the fix.
Full quic test suite: 295 tests, 0 failures, 0 errors.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Adds the client-side VN flow needed for the interop runner's
`versionnegotiation` testcase:
- `QuicConnection` accepts an `initialVersion` constructor parameter
(default `QuicVersion.V1`) and exposes a mutable `currentVersion`
the writer stamps into outbound long-headers. `start()` now caches
the ClientHello bytes for VN-driven re-emission.
- `applyVersionNegotiation(supportedVersions)` validates per §6.2
(anti-downgrade: reject if list contains the offered version),
picks v1 from the offered set, regenerates DCID, re-derives
Initial keys against the new DCID, resets the Initial level via
`LevelState.resetForVersionNegotiation`, re-enqueues the cached
ClientHello, and latches `vnConsumed` so a second VN is dropped.
Failure to find a mutually supported version closes the connection
with `QuicVersionNegotiationException`.
- `QuicConnectionParser.feedDatagram` detects `version == 0` long
headers BEFORE peekHeader (whose layout assumes v1) and dispatches
to a new `feedVersionNegotiationPacket` that parses the §17.2.1
shape and validates the echoed DCID.
- `QuicConnectionWriter` reads `conn.currentVersion` instead of the
hardcoded `QuicVersion.V1`.
- `QuicVersion.FORCE_VERSION_NEGOTIATION = 0x1a2a3a4a` for the
interop runner.
- `InteropRunner` honors `TESTCASE=versionnegotiation` (or
`-DinteropTestcase=`) and offers the force-VN version.
Regression coverage in `VersionNegotiationTest`:
- happy path: VN switches `currentVersion` to v1, regenerates DCID,
resets PN, and the next drain emits a v1 Initial on the wire.
- downgrade defense: VN listing the offered version is dropped.
- unsupported list: VN whose versions we can't speak fails the
handshake and closes the connection.
- second VN: post-consumption VN is ignored.
- DCID mismatch: spoofed VN with wrong echoed DCID is dropped.
- backward compatibility: default `initialVersion` keeps v1 behavior
for existing callers.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
When QuicConnection tears down (CONNECTION_CLOSE, read-loop death,
INTERNAL_ERROR from a saturated stream channel, etc.) the per-stream
incomingChannel objects were left open, so any application coroutine
suspended on `stream.incoming.collect { … }` hung forever waiting for
a FIN that would never come. The connection-wide signal channels
(closedSignal, peerStreamSignal, incomingDatagramSignal) all closed
cleanly, but the per-stream Flows did not — surfacing in the
quic-interop-runner `multiplexing` case as 677 collectors stuck after
the connection died mid-response, so zero of the 1999 expected files
landed.
Fix: closeAllSignals() now also calls closeIncoming() on every stream
in streamsList. Channel.close() is idempotent, and consumeAsFlow drains
already-buffered chunks before honouring the close, so any bytes the
parser had already pushed are still surfaced to the collector before
the Flow terminates.
Adds MultiStreamFinDeliveryTest covering: (a) FIN delivery to N parallel
client-bidi streams, (b) connection-teardown unblocks every per-stream
Flow, (c) buffered bytes survive a teardown without an explicit FIN.
The Retry parser + integrity-tag verifier already existed in
RetryPacket.kt, but feedDatagram dropped Retry packets on the floor.
Hook them up:
- QuicConnectionParser.feedLongHeaderPacket detects RETRY type before
the standard parse-and-decrypt path, parses via RetryPacket, and
dispatches to QuicConnection.applyRetry.
- QuicConnection.start() now caches the ClientHello bytes (TLS only
emits ClientHello once; we need to re-queue the same bytes on the
fresh Initial keys after Retry). New applyRetry method:
verifies the integrity tag, swaps DCID to Retry's SCID, re-derives
Initial keys, resets the Initial PN space + sentPackets +
cryptoSend, re-enqueues the cached ClientHello, stores the Retry
token, and latches retryConsumed so a second Retry is dropped.
- LevelState.restoreFromRetry / PacketNumberSpaceState.resetForRetry
give applyRetry an in-place reset (the level reference is a `val`,
so we mirror discardKeys' field-reset pattern).
- QuicConnectionWriter.buildLongHeaderFromFrames threads
conn.retryToken through the Initial header's Token field on every
Initial we emit after Retry.
Per RFC 9001 §5.8, a Retry with a bad integrity tag is silently
dropped; per RFC 9000 §17.2.5.2, only one Retry is honored per
connection. Both invariants are tested.
New test: RetryHandlingTest covers the happy path (DCID swap, PN
reset, token threading, ClientHello replay, ≥1200-byte padding),
the bad-tag path, and the second-retry path.
Pre-handshake PTO previously only set `pendingPing`, which collapsed to
either nothing (the bug 86b6c609a fixed in a sibling branch) or a bare
PING. Against aioquic in the quic-interop-runner ns-3 sim, the first
ClientHello can be dropped (server not ready at t≈0.5s); a follow-up
PING with the same DCID is silently ignored because the server has no
state for that DCID. We need to retransmit the actual ClientHello bytes
so the server sees a full connection attempt.
Implementation:
- SendBuffer.requeueAllInflight(): walks the inflight list and moves
every sent-but-not-ACK'd range to the retransmit queue, preserving
offset and FIN. Mirrors markLost's per-range path but applies to all
inflight ranges in one shot. Idempotent + best-effort safe.
- QuicConnection.requeueAllInflightCrypto(level): thin wrapper that
drives the per-level cryptoSend buffer's new method.
- QuicConnectionDriver.sendLoop: PTO branch now calls the new helper
at the highest active pre-application level (Handshake > Initial)
when 1-RTT keys aren't installed. The next drain naturally emits a
CRYPTO frame at the original offset (takeChunk drains the
retransmit queue first per existing semantics).
- QuicConnectionWriter.collectHandshakeLevelFrames: honors pendingPing
pre-handshake at the highest active level — but skips the PING when
a CRYPTO frame is already in the same level's frame list (the
CRYPTO retransmit covers the ack-eliciting requirement). Bare PING
still goes out when there's nothing to retransmit.
Old RecoveryToken.Crypto entries in sentPackets for the original PNs
remain harmless: when loss detection eventually declares them lost,
markLost re-runs against ranges that have already moved on, which is
itself idempotent (clamps to flushedFloor / no-op on already-queued).
Test: PtoCryptoRetransmitTest reproduces the wire scenario — first drain
emits ClientHello in a ≥1200-byte Initial datagram; simulated PTO
calls requeueAllInflightCrypto + sets pendingPing; second drain must
contain a CRYPTO frame at the same offset with the same payload bytes,
not a bare PING. Datagram size still ≥1200 (RFC 9000 §14.1).
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
The padding-rebuild branch in QuicConnectionWriter.drainOutbound computed
`padBytes = 1200 - natural`, but the QUIC long-header Length field is a
varint (RFC 9000 §16). When the natural-size payload was small enough for
Length to fit in 1 byte (body ≤ 63 bytes), the rebuild's larger body
crossed the 64-byte threshold and Length grew to 2 bytes — adding 1 wire
byte that wasn't in `natural`. PING-only PTO probe Initials therefore went
out at exactly 1199 bytes, one short of the §14.1 floor.
Fix: rebuild iteratively. After the first rebuild, measure the actual
datagram size; if still < 1200, bump padBytes by the residual and rebuild
once more. PADDING bytes inside the AEAD envelope add 1:1 to the wire
size and the Length varint grows monotonically, so the loop terminates
in ≤ 2 iterations for any reachable payload.
Same fix is applied to buildClosingDatagram so close-only Initial probes
on the boundary aren't tripped by future varint-growth changes.
Tightens the existing PTO-probe regression test to assert ≥ 1200 (was
relaxed to ≥ 1199 in 86b6c609a) and adds a new boundary test that builds
a single-byte-payload Initial and checks 1200 ≤ size ≤ 1203 — strict
floor with a tight ceiling so over-correction would also fail.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
The aioquic interop run revealed bug #3 (after the close-padding and
close-frame-type fixes): when PTO fires before the handshake completes,
the driver sets `pendingPing = true` but the writer only consumed that
flag in the 1-RTT path. Pre-handshake the flag was silently discarded,
so the second drain produced no Initial datagram — the connection sat
mute through every subsequent PTO. Symptom on the wire: exactly one
Initial packet (the close at PN=1, after our internal handshake
timeout), zero retransmits across the full 10-second budget, no chance
for the peer to recover from a dropped first ClientHello.
Fix routes pendingPing through to whatever encryption level is the
highest currently active — preferring 1-RTT, falling through Handshake,
finally Initial. Adds a regression test that drains a fresh connection
with `pendingPing = true` and verifies an Initial-level padded probe
datagram comes out (vs. null pre-fix).
Test relaxes the size assertion to ≥ 1199 due to a separate pre-existing
off-by-one in the writer's padding deficit calculation when the natural
payload uses a 1-byte Length varint that grows to 2 after padding —
that's a follow-up; the regression we care about here is "no probe at
all," not the byte-precise padding edge.
Outstanding from this run:
- Strict ≥ 1200 padding for tiny payloads (PING-only Initial = 1199)
- PTO should retransmit unacked CRYPTO bytes, not just emit a PING
(current PING gets ACK + relies on packet-number-threshold loss
detection to trigger CRYPTO retransmit; works but suboptimal)
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
The quic-interop-runner against aioquic surfaced two real bugs in the
writer's CLOSING-status branch, both visible on the wire as a single
~45-byte UDP datagram instead of a properly framed close.
§10.2.3 — at Initial / Handshake levels only CONNECTION_CLOSE (Transport,
0x1c) is allowed. The application-level form (0x1d) leaks app state
pre-handshake. The writer was unconditionally building a 0x1d frame and
shipping it inside an Initial packet; aioquic dropped it silently.
§14.1 — any client datagram containing an Initial MUST be ≥ 1200 bytes
in UDP-payload terms. The CLOSING branch bypassed the existing padding
logic, so close-only Initial datagrams went out at ~45 bytes and
servers correctly rejected them (also as malformed).
Fix replaces the CLOSING branch with a dedicated `buildClosingDatagram`
helper:
- Application keys present → 0x1d, original error code + reason.
- Pre-1-RTT (Handshake or Initial) → 0x1c with errorCode =
APPLICATION_ERROR (0x0c), frameType=0, empty reason.
- Initial level: build at natural size, rewind PN if < 1200, rebuild
with PADDING-frame deficit inside the AEAD envelope.
Plus a regression test covering both: pre-handshake close datagram size
≥ 1200, and ConnectionCloseFrame round-trips 0x1c vs 0x1d for the right
constructor inputs.
NOT addressed yet: why our ClientHello at PN=0 doesn't appear in the
runner pcap (only PN=1 close does). With this fix the close packet is
now well-formed; the next runner run will tell us whether the missing
ClientHello is a sim/capture artifact or a separate writer bug.
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
Adds two debugging hooks to :quic so the interop endpoint can produce
artifacts that make the runner actually useful for finding bugs:
- TlsClient.clientRandom: capture and expose the 32-byte ClientHello
random so a SSLKEYLOG line can correlate to this connection.
- QuicConnection.extraSecretsListener: optional chained secrets
listener (default null, no-op for production callers). Fires
alongside the connection's own key-installation listener at every
encryption-level transition.
- QuicConnection.cipherSuites: knob to override the offered TLS
cipher suites in ClientHello.
InteropClient now:
- Writes NSS Key Log Format lines to $SSLKEYLOGFILE when set, so
Wireshark can decrypt the sim's pcap captures.
- Implements the `chacha20` testcase by offering only
TLS_CHACHA20_POLY1305_SHA256 — exercises the ChaCha20 AEAD path
end-to-end against a peer.
Defers QLOGDIR (needs qlog observer infrastructure across packet/
frame/recovery layers — own design doc) and `versionnegotiation`
(needs the writer to accept a configurable initial QUIC version).
https://claude.ai/code/session_01HcvfQq1ttPV9PkRoJb4nyT
The previous priority-then-round-robin shape applied the rotating
start-index globally over the sorted list, so the cross-tier order
flipped on alternate drains: drain N had high-priority first, drain
N+1 advanced streamRoundRobinStart and had low-priority first. The
priority hint was silently defeated under any sustained traffic.
Replace it with strict priority across tiers + round-robin only
within each same-priority tier. Higher tiers always drain ahead of
lower ones; same-priority peers continue to take turns via the
existing rotating start. Default-priority callers see no behaviour
change (single tier, identical rotation semantics).
Tighten the test: drain twice and assert the higher-priority stream
emits first on BOTH drains — the regression case that the single-
drain version of the test missed. Add a regression guard for the
same-priority round-robin so a future refactor can't silently
serialise on the first stream.
https://claude.ai/code/session_01KWdr4RjVvyYZfEuPVaQfUa
Bias the QUIC connection writer's drain loop toward higher-priority
streams so moq-lite group streams with newer (higher) sequence numbers
drain ahead of older ones under congestion. Implements the T11.3
follow-up flagged in nestsClient/plans/2026-05-06-stream-priority-
followup.md (now removed).
QuicStream gets a `@Volatile var priority: Int = 0`. The writer's
streamsView iteration is replaced by a stable sortedByDescending pass
so same-priority streams keep their existing rotating-start round
robin while higher-priority tiers always drain first.
WebTransportWriteStream gains a `setPriority(Int)` hook; the QUIC-
backed adapter forwards to the underlying QuicStream, while the
in-memory test fakes treat it as a no-op. MoqLiteSession.openGroupStream
calls `uni.setPriority(sequence)` (saturating to Int.MAX_VALUE) to
mirror moq-rs's `Publisher::serve_group`.
Tests: a new InMemoryQuicPipe.decryptClientApplicationFrames helper
walks past coalesced long-header packets to surface 1-RTT frames,
which lets QuicConnectionWriterTest assert that the higher-priority
stream's StreamFrame lands first inside a single drained packet.
https://claude.ai/code/session_01KWdr4RjVvyYZfEuPVaQfUa
ShortHeaderPacket.build / LongHeaderPacket.build crashed with
`IllegalArgumentException: packet too short for HP sample` whenever the
plaintext payload was small enough that pnLen + payload < 4 — most
visibly on the 1-RTT path when buildApplicationPacket emitted a single
1-byte PING (PTO probe with no ACKs queued and no streams to drain) and
the packet number still fit in 1 byte. The crash tore down the writer
loop, which surfaced upstream as moq-lite "subscribe stream FIN before
reply" because in-flight bidi streams got FIN'd by the peer.
RFC 9001 §5.4.2 mandates the sender pad the plaintext so the encrypted
output (plaintext + 16-byte AEAD tag) has at least 20 bytes after the
packet-number offset for the 16-byte HP sample. The fix pads the
plaintext with trailing 0x00 bytes — those decode as PADDING frames per
RFC 9000 §19.1 and decodeFrames already absorbs them. For long-header
packets the padded size feeds back into the Length varint.
After drafting the congestion-control plan we concluded the audio-rooms
workload doesn't actually need CC — speakers push ~8 KB/sec, which
never fills any modern link's capacity. The one real concern that
surfaced — STREAM retransmit wasting bandwidth on stale Opus frames
on lossy uplinks — is much cheaper to fix directly than to bound via
a 14-test CC subsystem.
SendBuffer gains a `bestEffort: Boolean = false` constructor flag.
When true, markLost drops the lost ranges instead of moving them to
the retransmit queue and lets the underlying byte storage compact as
if the bytes had been ACK'd. The FIN flag (if covered) also stays
sent — best-effort skips FIN re-emission too. The peer may end up
with a truncated stream; moq-lite's per-stream timeouts handle that.
Plumbed through QuicStream → QuicConnection.openUniStream(bestEffort)
→ QuicWebTransportSessionState.openUniStream(bestEffort) →
WebTransportSession.openUniStream(bestEffort). Default is false
everywhere, so reliable streams (HTTP/3 control, moq-lite SUBSCRIBE
bidi, etc.) keep RFC 9000 §3.5 semantics.
MoqLiteSession.openGroupStream now passes `bestEffort = true` —
group streams carry a single Opus packet, are real-time, and don't
benefit from retransmit.
Internal cleanup: `removeOverlap`'s `ackedNotLost: Boolean` parameter
became `OverlapAction { ACK, RETRANSMIT, DROP }` so the third best-
effort disposition has a name. Same code paths, same tests, just
clearer at the call site.
CC plan (quic/plans/2026-05-05-congestion-control.md) is updated to
"parked indefinitely" with a note that this commit is the lighter-
weight alternative that addresses the only practical concern. The
plan is preserved as a reference if a future workload justifies CC.
New tests: SendBufferBestEffortTest (6 cases — reliable baseline,
best-effort drops, FIN drop in best-effort mode, partial overlap,
idempotent stale loss, ACK path still works).
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Pre-fix `:quic` held Initial AND Handshake encryption-level state
indefinitely once derived. AEAD cipher state, per-level CRYPTO
buffers, and the per-level sent-packet map all stayed alive for the
lifetime of the connection — a real memory leak for long sessions
(audio rooms run for hours).
LevelState.discardKeys() (idempotent):
- Nulls sendProtection / receiveProtection (frees AEAD state).
- Replaces cryptoSend / cryptoReceive with empty instances.
- Replaces ackTracker with an empty instance.
- Clears sentPackets and resets largestAckedPn /
largestAckedSentTimeMs.
- Latches keysDiscarded = true.
Hook locations:
- Initial discard (RFC 9001 §4.9.1, client side): in
QuicConnectionWriter.drainOutbound, after a Handshake-level packet
is built into the outbound datagram. The next drainOutbound MUST
NOT touch the Initial level; any retransmitted Initial from the
peer is silently dropped (receiveProtection == null), which is
correct per the same RFC since the server has also moved up
encryption levels by then.
- Handshake discard (RFC 9001 §4.9.2 + §4.1.2, client side): in
QuicConnectionParser, on receipt of a HANDSHAKE_DONE frame.
Once a level's protection is null, parser-side decrypt at that level
returns null silently (existing receiveProtection == null check) and
writer-side build skips it (existing sendProtection == null check),
so no further code paths needed updating.
New test: KeyDiscardTest (4 cases — Initial keys discarded after
first Handshake packet, Handshake keys still live until
HANDSHAKE_DONE, Handshake keys discarded on HANDSHAKE_DONE,
discardKeys is idempotent).
Listed in the audit-summary deferred-work as item 3
(`No Initial / Handshake key discard`).
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Pre-fix, QuicConnectionParser purged the inbound AckTracker on every
inbound AckFrame using `frame.largestAcknowledged - frame.firstAckRange`
— but that value lives in OUR outbound PN space, while the tracker
holds inbound PNs we received from the peer. The two PN spaces are
unrelated; the bug mostly hid because they grow at similar rates,
but caused range-list bloat over long sessions where traffic is
asymmetric (e.g. listener receives ~50 audio frames/sec while sending
back ~1 ACK/sec).
The correct semantics: purge only when the peer has confirmed receipt
of OUR outbound ACK frame. Now driven by the ACK-of-ACK dispatch.
- RecoveryToken.Ack changed from data object to data class carrying
(level, largestAcked) — the encryption level and the largest inbound
PN our outbound ACK frame covered.
- QuicConnectionWriter populates these fields from the AckFrame at
emit time.
- QuicConnection.onTokensAcked dispatches RecoveryToken.Ack to
levelState(level).ackTracker.purgeBelow(largestAcked + 1).
- The wrong purge in QuicConnectionParser is removed (replaced with a
comment pointing at the new dispatch path).
Listed in the audit-summary deferred-work as item 6
(`AckTracker.purgeBelow threshold semantics`).
New test: AckTrackerPurgeOnAckOfAckTest (4 cases — purge on
ack-of-ack, level routing, partial purge keeps higher PNs, out-of-order
ACKs are safe).
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Audit follow-up from the prior two commits.
1. resetStream() / stopSending() now no-op on the second call. RFC 9000
§3.5 pins finalSize at first emission; replaying retransmits with a
larger value (because the app enqueued more bytes between two
resetStream calls) would trigger FINAL_SIZE_ERROR on the peer. The
"idempotent: a second call overwrites with the newer error code"
claim was simply wrong. Two new tests lock the contract:
resetStream_secondCallIsNoOp_finalSizeFrozen and
stopSending_secondCallIsNoOp.
2. resetEmitPending / resetAcked / stopSendingEmitPending /
stopSendingAcked are now @Volatile. The public emit APIs are
callable from any coroutine while the writer / loss / ACK
dispatchers read the same fields under QuicConnection.lock; volatile
gives the cross-thread happens-before, and the first-call-wins gate
above eliminates the only multi-writer race (two app threads racing
the writer's clear-after-emit).
3. SendBuffer's class-level KDoc still claimed range arithmetic was
O(N) "swap to TreeMap if profiling flags it" — stale after the
binary-search refactor in 303caa8. Updated to reflect the actual
O(log N + k) cost.
4. The bulk-removal comment in removeOverlap overstated the win
("O(k) per call, single shift of trailing entries"). ArrayDeque
removeAt(i) shifts on every call, so the actual cost is
O(k * (size - end + k)). Toned down — it's still cheap because
k is 1-2 in steady state.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
removeOverlap was O(N) on the in-flight list — every ACK or loss
notification scanned the whole deque. Replaced with a binary-search
firstOverlapIndex helper plus a forward early-exit walk and a backward
bulk-removal pass. addToInFlight likewise binary-searches for the
middle-insert position instead of linear-scanning.
The list is sorted by offset and non-overlapping by construction, so
firstOverlapIndex finds the first entry whose end-offset > target in
O(log N), and the walk terminates as soon as r.offset >= rangeEnd.
Workload today is small (<100 entries per stream), but audio rooms
with many active streams compound the per-ACK cost.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Application code can now call QuicStream.resetStream(errorCode) and
stopSending(errorCode); the next writer drain emits the matching frame
with a RecoveryToken. Loss dispatch re-flags the per-stream emit-pending
bit; ACK dispatch latches resetAcked / stopSendingAcked so stale loss
notifications can't re-emit. NEW_CONNECTION_ID retransmit drains
QuicConnection.pendingNewConnectionId on next writer pass (no public
emit API since :quic doesn't rotate connection IDs, but the wiring is
in place for a future emit path).
Five tests in ResetStopSendingEmitTest mirror neqo's send-stream reset
coverage: emit-and-token, retransmit-on-loss, ack-then-stale-loss-drop,
stop-sending emission, and NEW_CONNECTION_ID drain.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Closes commits D + E of the deferred-follow-ups pass. With
SendBuffer's retain-until-ACK semantics in commit B and the
Crypto / ResetStream / StopSending / NewConnectionId tokens added
in commit A, the writer now records a Crypto token per CRYPTO
frame at each encryption level (Initial, Handshake) so RFC 9002
retransmit recovers lost handshake bytes.
# Writer side
QuicConnectionWriter.collectHandshakeLevelFrames returns a new
HandshakeLevelContents(frames, tokens) pair instead of a bare
frame list. For each emitted CryptoFrame it appends a matching
RecoveryToken.Crypto(level, offset, length).
QuicConnectionWriter.buildLongHeaderFromFrames now also takes the
parallel tokens list, and after encryption records a SentPacket
in the matching LevelState.sentPackets map. Initial-level rebuilds
with padding (RFC 9000 §14.1) call pnSpace.rewindOutboundForRebuild
to reuse the same PN — the second build's map insert overwrites
the prior entry, so retention reflects the final padded packet.
drainOutbound's two callsites updated to pass tokens through.
# ACK / loss dispatch
Already wired in commit A. QuicConnection.onTokensAcked routes
Crypto tokens to LevelState.cryptoSend.markAcked at the matching
level, releasing buffer memory as the contiguous low end is ACK'd.
QuicConnection.onTokensLost routes them to markLost, re-queueing
the bytes for retransmit at the same level.
# RESET_STREAM / STOP_SENDING / NEW_CONNECTION_ID
Same dispatcher-only completion. The pendingResetStream /
pendingStopSending / pendingNewConnectionId maps on QuicConnection
are populated by the loss dispatcher when those token types are
seen. :quic doesn't currently emit any of those frames (no
application code triggers stream reset, connection-ID rotation
isn't wired), so the writer never drains the maps yet —
scaffolding for future emit support. The exhaustive when in
onTokensLost / onTokensAcked is now complete: any future addition
of a new RecoveryToken variant trips the compile-time exhaustive
check, mirroring the test in RecoveryTokenTest.
# Tests added (3, all pass)
CryptoRetransmitTest:
- handshakePacket_carriesCryptoToken_inSentPacket: ClientHello
emission produces an Initial-level SentPacket with a Crypto
token at offset 0 with the expected level.
- cryptoData_lostAndRetransmittedAtSameLevel: simulate loss via
direct dispatch, observe re-queue in cryptoSend, verify next
drain produces a fresh Initial packet replaying the same
offset (RFC 9000 §13.3 idempotent).
- cryptoAck_releasesBufferAtSameLevel: ACK via onTokensAcked,
cryptoSend's readableBytes drops to 0.
# Net result
Lost handshake bytes (ClientHello, EncryptedExtensions, Certificate,
Finished, NewSessionTicket) are now recovered automatically. The
prior 1-second-fixed-PTO placeholder in QuicConnectionDriver
(commit step 7 of the prior plan) becomes meaningfully more useful
— PTO now wakes the writer to retransmit ACTUAL data, not just
emit empty PINGs.
Full :quic test suite + nestsClient moq-lite + amethyst Android
compile all pass.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Closes commit C of the deferred-follow-ups pass. With the
SendBuffer rewrite in commit B (retain-until-ACK with markAcked /
markLost), the connection now wires STREAM frames into the same
RFC 9002 retransmit path that already handles flow-control
extensions.
# Writer side
QuicConnectionWriter.buildApplicationPacket records a
RecoveryToken.Stream(streamId, offset, length, fin) for every
STREAM frame it emits. The token captures the on-wire byte range
plus the FIN bit so retransmit can reproduce the same StreamFrame
on next drain.
# ACK side
New QuicConnection.onTokensAcked() mirrors onTokensLost. The
parser's AckFrame handler iterates the drained packets and routes
each to onTokensAcked, which:
- For Stream tokens: calls SendBuffer.markAcked(offset, length).
The buffer removes the range from in-flight; if the contiguous
low end is now fully ACK'd, flushedFloor advances and storage
shifts forward.
- For Crypto tokens: same shape, applied to the per-level
cryptoSend buffer (commit E will exercise this path for
handshake reliability — Crypto retransmit is wired now but the
writer's CRYPTO emission path doesn't yet record Crypto tokens;
that's commit E).
- For control-frame and Ack tokens: ACK-no-op. The frame already
did its job by reaching the peer; no per-buffer state to
release.
# Loss side
onTokensLost (commit A) already routes Stream tokens to
SendBuffer.markLost. With commit B's real implementation (was a
no-op stub), this now actually re-queues the byte range for
retransmit. The next writer drain pulls from the retransmit queue
before any fresh sends, with the original offset preserved (RFC
9000 §13.3 idempotent retransmit).
# Tests added (3, all pass)
- streamFrame_carriesStreamToken_inSentPacket: writer emits a
Stream token whose fields match the StreamFrame on the wire
- streamData_lostAndRetransmittedOnNextDrain: simulate loss via
direct dispatch, observe re-emit at the same offset in a fresh
SentPacket (different PN)
- streamData_ackedReleasesBuffer: ACK via onTokensAcked,
enqueue more bytes, observe the next send picks up at the
post-ACK offset (proves bytes were released and floor advanced)
Full :quic test suite, nestsClient moq-lite tests, amethyst Android
compile all pass.
Net result: lost STREAM data (e.g. nestsClient bidi control-stream
bytes — moq-lite Subscribe/Announce control messages travel on
QUIC bidi streams) is now recovered automatically. Audio rooms
benefit indirectly: the relay's announce/subscribe path is more
resilient to packet loss.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Foundation for STREAM and CRYPTO retransmit (commits C, D of the
deferred-follow-ups pass). Replaces the prior best-effort mode
where takeChunk released bytes immediately.
# Three logical regions
The buffer covers `[flushedFloor, nextOffset)`. Each byte is in one
of three states:
- In-flight: sent but not yet ACK'd. Sorted-by-offset list.
- Needs retransmit: declared lost; re-sent before any fresh bytes.
FIFO queue.
- Unsent: `[nextSendOffset, nextOffset)`.
# New API
- markAcked(offset, length): removes the range from in-flight,
advances flushedFloor through any contiguous low-end ACKs and
shifts the underlying byte storage forward. Length=0 means
FIN-only ACK; latches finAcked = true.
- markLost(offset, length, fin): removes from in-flight, appends
to retransmit queue. If fin, clears finSent so the FIN gets
re-emitted. Idempotent: stale loss notifications below
flushedFloor are absorbed.
takeChunk priority order:
1. retransmit queue (preserves original offset; same byte data)
2. fresh unsent bytes from [nextSendOffset, nextOffset)
3. FIN-only zero-byte chunk if finPending and everything drained
# Range arithmetic
removeOverlap walks in-flight, computes the three-piece split for
each overlapping range (leftKept, covered, rightKept) and either
drops the covered portion (ACK) or pushes it onto retransmit
(loss). FIN belongs to the rightmost piece, so split at the end of
the original range correctly preserves it.
# Compaction
advanceFlushedFloorIfPossible bumps the floor whenever the lowest
in-flight / retransmit / unsent offset is above it, then
ByteArray.copyInto shifts the data window forward. Memory bounded
by `nextOffset - flushedFloor` rather than growing unboundedly.
# FIN handling
Treated as a virtual byte at offset = nextOffset. finPending arms
takeChunk to attach FIN to the final data chunk; finSent latches
true on emission; markLost(fin=true) clears it for re-emission;
finAcked latches true when the FIN-bearing range is ACK'd.
# Tests added (14, all pass)
- takeChunk_releasesNothingUntilAcked
- markAcked_full_releasesBytes_andDoesNotResend
- markLost_movesBytesToRetransmitQueue_takeChunkReplaysSameOffset
- markLost_partialRange_splitsInFlight
- markAcked_partialRange_splitsInFlight
- retransmitDrainsBeforeFreshBytes (priority order)
- maxBytesSplits_acrossRetransmitAndFresh
- fin_carriedOnFinalDataChunk_andRetransmittedOnLoss
- fin_only_emittedAfterDataDrained
- fin_only_lostAndRetransmits
- markAcked_advancesFlushedFloor_releasesMemory
- markAcked_outOfOrder_preventsFloorAdvance
- markLost_belowFlushedFloor_isNoop (defensive)
- finAcked_latchesTrueOnce
- readableBytes_reflectsRetransmitPlusFresh
- multipleSendsWithinSingleEnqueue_acksIndependently
Existing :quic tests (handshake, flow control, frame routing) +
nestsClient moq-lite + amethyst Android compile all pass.
SendBufferConcurrencyTest unchanged — the synchronized-on-this
discipline carries over from the prior implementation.
Wires nothing yet — commits C/D will route Stream/Crypto tokens
through markLost. The dispatcher in QuicConnection.onTokensLost
already calls markLost (added in commit A as a no-op stub); now
those calls actually do work.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Token-shape commit (commit A of the deferred-follow-ups pass that
extends RFC 9002 retransmit beyond the receive-side flow-control
extensions originally shipped at commit 9e6fa3d).
New variants on RecoveryToken sealed class:
- Stream(streamId, offset, length, fin): STREAM data range we
sent. On loss the dispatcher will re-queue the range on the
stream's SendBuffer (commits C, D wire that).
- Crypto(level, offset, length): CRYPTO bytes at a specific
encryption level. RFC 9000 §17.2.5 + §13.3: handshake bytes
are reliable; lost CRYPTO retransmits at the same encryption
level. Commit E wires the per-level dispatch.
- ResetStream(streamId, errorCode, finalSize),
StopSending(streamId, errorCode),
NewConnectionId(seq, retirePriorTo, cid, statelessResetToken):
reliable per RFC 9000 §13.3, scaffolding-only since :quic
doesn't currently emit any of these. The pendingResetStream /
pendingStopSending / pendingNewConnectionId maps on
QuicConnection are populated by the dispatcher but not yet
drained by any writer code path.
QuicConnection.onTokensLost extended with the new variants:
- Stream/Crypto: route to the matching SendBuffer.markLost
(currently a no-op — see SendBuffer.markLost kdoc; commit C
replaces the stub with the retain-until-ACK rewrite).
- ResetStream/StopSending/NewConnectionId: write to the
pending* maps; future emit code drains them.
NewConnectionId data-class needs explicit equals/hashCode because
its ByteArray fields would otherwise compare by identity (Kotlin
data-class auto-equals limitation). Tested.
Tests added (4):
- whenDispatch_isExhaustive extended with all 10 variants;
catches at compile time if a new variant is added without
updating the dispatcher
- newConnectionId_arrayEqualityIsByContent
- stream_equalityByValue
- crypto_equalityByValue
SendBuffer gains placeholder markLost(offset, length, fin) and
markAcked(offset, length) methods — both no-ops today. Their
signatures are stable so the dispatcher wiring lands once now and
doesn't need re-touching when commit C makes them functional.
Full :quic test suite passes.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Claude