Replace the wall-clock dance with a direct EventStoreProjection
construction wired to a controllable nowProvider. Both events are
inserted with expirations safely in the future so the SQL
reject_expired_events trigger never fires; the test then bumps the
fake clock and replays the StoreChange.DeleteExpired the observable
would emit after a sweep. Tests the projection's reaction to the
sweep signal — which is what this case was always about — without a
6s real-time delay or a flake window. Full SQL sweep behavior
remains covered by ExpirationTest.testDeletingExpiredEvents.
The reject_expired_events trigger checks NEW.expiration <= unixepoch()
at insert time, so signing/insert latency on a slow runner could push
unixepoch() to time+1 before the row was committed, raising
SQLiteException at runBlocking entry. Bumping the short event to
time+5 with a matching delay(6000) mirrors
ExpirationTest.testDeletingExpiredEvents and gives the trigger room
to breathe without changing the behavior under test.
Headline numbers from a fresh `./gradlew assemble`:
- :commons material_symbols_outlined.ttf 11M -> 409K (subset to the 210
codepoints actually referenced from MaterialSymbols.kt)
- :amethyst per-ABI APKs no longer built in CI (-PdisableAbiSplits=true);
~600M of stripped_native_libs intermediates skipped per run
Changes:
- .git-hooks/pre-push runs only :amethyst:testPlayDebugUnitTest plus jvmTest
on the KMP modules instead of `./gradlew test`, which was compiling all six
amethyst variants (play/fdroid x debug/release/benchmark)
- amethyst/build.gradle adds three opt-in fast-build flags:
-PdisableAbiSplits=true skip per-ABI APK splits
-PdisableUniversalApk=true skip the universal APK output
-Pamethyst.skipMapping=true disable R8 (release+benchmark)
Defaults are unchanged; release pipelines must not set skipMapping.
- .github/workflows/build.yml uses gradle/actions/setup-gradle@v4 with
cache-read-only on PRs / cache-write on main, drops --no-daemon, collapses
the Android job to a single Gradle invocation (lint + focused debug unit
tests + assembleBenchmark with -PdisableAbiSplits=true), and globs both
APK naming patterns for the upload step.
- tools/material-symbols-subset/{subset.sh,README.md} regenerates the font
from upstream + MaterialSymbols.kt; run after adding/removing icons.
Verified: ./gradlew :amethyst:help with all three new -P flags parses cleanly,
and ./gradlew :commons:jvmJar --rerun-tasks succeeds with the subsetted font
(commons-jvm jar shrinks from ~13M to 2.9M).
https://claude.ai/code/session_01YSmkagXXN5AwGcY4upiUh1
The test emitted FRAME1 into first.frames and immediately called
first.fail(...), trusting that FRAME1 would propagate through the pump
to the wrapper's frames before the session swap. emit() is
non-suspending — it only enqueues FRAME1 into the pump's slot. On
slower hosts (observed on macOS CI) the orchestrator's reconnect path
can flip activeListener to the second session before the pump's
collect lambda runs, at which point collectLatest cancels
pump-iteration-1 mid-resume and FRAME1 is dropped. The consumer's
take(2) then only ever sees FRAME2 and the async's withTimeout fires
after 5 s.
Add a consumerProgress StateFlow that the consumer's collector bumps
on each frame, and wait for it to reach 1 before failing the listener.
Same shape as the existing consumerSubscribed gate that closed the
"emit before consumer subscribes" race.
The -PskipVlcSetup / AMETHYST_SKIP_VLC opt-out is no longer used: CI
relies on the actions/cache step in build.yml to avoid hitting
get.videolan.org, and the in-build retry budget covers cache misses.
Remove the dead opt-out from desktopApp/build.gradle.kts and the env
export from the session-start hook.
Replace the -PskipVlcSetup=true CI bypass with a per-OS GitHub Actions
cache of ~/.gradle/vlcSetup. Cache key is hashFiles('desktopApp/build.gradle.kts')
so a VLC version bump invalidates cleanly; restore-keys lets unrelated
edits to that file still warm-start.
Cache hit: vlcDownload / upxDownload are up-to-date and we never touch
get.videolan.org. Cache miss (first run after version bump, or a new
runner): we fall back to the network, where the existing 5-attempt
retry budget in build.gradle.kts already handles flakes.
The skipVlcSetup property + AMETHYST_SKIP_VLC env hooks stay in
desktopApp/build.gradle.kts as a local-dev / CCW escape hatch — the
CCW egress can't reach get.videolan.org at all and has no actions/cache.
Disabling only vlcDownload/upxDownload/vlcSetup left the chained
*Extract tasks in the graph with @InputFile properties pointing at
archives that were never downloaded — Gradle 9 then fails the build
during input validation:
property 'upxArchiveFile' specifies file
'/root/.gradle/vlcSetup/upx-4.2.4.tar.xz' which doesn't exist.
Disable the full vlc-setup plugin task set
(vlcDownload, vlcExtract, vlcFilterPlugins, vlcCompressPlugins,
upxDownload, upxExtract, vlcSetup) so packaging skips VLC bundling
cleanly.
Verified locally: :desktopApp:packageDeb now proceeds past those tasks
straight to jpackage with -PskipVlcSetup=true.
The previous signature took Event and runtime-checked kind == 9 inside
the notifier. That left the contract implicit: any caller could pass a
reaction or control message and the call would silently no-op.
Type the parameter as ChatEvent so the kind:9 restriction is structural,
and move the narrowing (`is ChatEvent`) to the GroupEventHandler call
site where the inner event is parsed. Reactions, deletions, and other
inner kinds now can't reach the notifier in the first place.
Drops the runtime kind check and the now-stale comment about reactions.
get.videolan.org regularly times out from GitHub-hosted runners and
Claude Code web sandboxes, taking the desktop CI build down with it
even though the failure is unrelated to the change under review.
Wire a -PskipVlcSetup=true (env: AMETHYST_SKIP_VLC=true) opt-out in
desktopApp/build.gradle.kts that disables the vlcDownload, upxDownload,
and vlcSetup tasks. Pass it from build.yml so PR CI no longer gates on
VLC reachability, and export the env from the CCW session-start hook.
Release packaging (create-release.yml) intentionally does not set the
flag, so shipped artifacts still bundle VLC.
Welcomes (kind:444) already had a direct-dispatch path because they have
no `p` tag and the cache-observer route can't match them to an account.
Group messages (kind:445) have the same problem — recipients are routed
by the `h` tag carrying the nostr_group_id — but were silently missed,
so the user only saw a popup when first added to a group, never for any
chat that followed.
Mirror the notifyWelcome path: after MarmotInboundProcessor decrypts
and verifies an ApplicationMessage and we've persisted the inner event
for the first time, fire notifyGroupMessage on NotificationDispatcher.
The notifier filters to ChatEvent (kind:9) so reactions, deletions and
control messages stay silent (matching how NIP-17 only notifies on
kind:14), applies the same 15-min freshness and self-author gates as
the other DM paths, and uses the marmot:<groupHex>?account=<npub>
deep-link scheme so taps land in the right chatroom.
Only fires on first-time decryption (isNew) so a relay re-broadcast or
on-disk persist replay can't double-notify.
The single-pane chat path wraps content in DisappearingScaffold, which applies
Modifier.imePadding() at the root surface. The dual-pane MessagesTwoPane uses a
plain Scaffold with no IME inset handling, so when the keyboard opens the
PrivateMessageEditFieldRow stayed hidden behind it.
https://claude.ai/code/session_012sPZ9KbbkTzdPL2KTn29ak
CreateNestViewModel is keyed by user pubkey via `viewModel(key = …)`,
so the same instance survives across sheet open/close cycles. After a
successful publish the form was left untouched: room name, summary,
image URL stayed populated and — most visibly — `isPublishing` was
never cleared, so on the second open the Submit button rendered as a
spinning progress indicator forever.
Reset the form to fresh defaults (re-seeded from the user's saved
kind-10112 list) at the end of `publishAndBuildLaunchInfo()`, right
before returning the launch info. The sheet is about to dismiss
anyway, so the user never sees the in-place reset; the next open
behaves like a first open.
ShortHeaderPacket.build / LongHeaderPacket.build crashed with
`IllegalArgumentException: packet too short for HP sample` whenever the
plaintext payload was small enough that pnLen + payload < 4 — most
visibly on the 1-RTT path when buildApplicationPacket emitted a single
1-byte PING (PTO probe with no ACKs queued and no streams to drain) and
the packet number still fit in 1 byte. The crash tore down the writer
loop, which surfaced upstream as moq-lite "subscribe stream FIN before
reply" because in-flight bidi streams got FIN'd by the peer.
RFC 9001 §5.4.2 mandates the sender pad the plaintext so the encrypted
output (plaintext + 16-byte AEAD tag) has at least 20 bytes after the
packet-number offset for the 16-byte HP sample. The fix pads the
plaintext with trailing 0x00 bytes — those decode as PADDING frames per
RFC 9000 §19.1 and decodeFrames already absorbs them. For long-header
packets the padded size feeds back into the Length varint.
Audit of the four prior commits found two genuine regressions and two
robustness gaps in the new code paths.
Bug A: NestForegroundService.networkCallback dropped the publish for
the most important scenario it was added to handle. The earlier guard
`if (previous != null && previous != network)` suppressed both the
registration-time first onAvailable AND the legitimate WiFi-loss-
then-cellular-available path. On a WiFi → cellular handover the
sequence is `onLost(wifi)` (clears currentDefaultNetwork to null)
followed by `onAvailable(cellular)` — which then looks identical to
the registration callback to the guard, so no publish fires and the
QUIC session sits on the dead socket until PTO. Replaced the implicit
"previous == null" suppression with an explicit `seenInitialNetwork`
flag that's set true on the first onAvailable and never cleared, so
post-onLost reconnects publish correctly.
Bug B: requestAudioFocus result handling was too permissive. The
shape `if (result == AUDIOFOCUS_REQUEST_FAILED) TransientLoss else
Granted` falls through to Granted on the runCatching exception path
(`result == null`) and on AUDIOFOCUS_REQUEST_DELAYED (= 2) — meaning
audio plays over an active call when the OS hasn't actually released
focus. Switched to a strict `if (result == AUDIOFOCUS_REQUEST_GRANTED)`
check; everything else (FAILED, DELAYED, exception) starts the VM
muted.
Robustness: NestViewModel.openSubscription's onError callback used to
swallow every AudioException, which fit the per-packet decoder-error
case but turned PlaybackFailed and DeviceUnavailable from a deferred
beginPlayback into a permanent "Connecting…" spinner on the speaker
tile. Now we discriminate by AudioException.Kind: decoder/encoder
errors stay swallowed (Opus PLC papers them over), but PlaybackFailed
and DeviceUnavailable roll the slot back so a future reconcile can
retry.
Pre-roll ordering swap + tests: NestPlayer.play used to call
beginPlayback BEFORE flushing the pre-roll buffer, leaving a
microsecond window where the AudioTrack hardware was playing against
an empty buffer. AudioTrack MODE_STREAM explicitly supports write()
before play(), so flush-then-beginPlayback is the textbook pattern
and what the fix now does. Three regression tests cover:
- pre-roll defers beginPlayback until threshold is met (and the
flush-then-begin ordering is observable)
- partial pre-roll flushes when the upstream flow ends early
- empty flow doesn't begin playback at all
The FakeAudioPlayer grows beginPlaybackCount + queuedAtBeginPlayback
fields so the tests can assert ordering directly.
Four follow-up fixes from the post-audit review (#4 / #5 / #6 / #7).
#6 AcousticEchoCanceler / NoiseSuppressor / AGC on the AudioRecord
session. The VOICE_COMMUNICATION input source engages the platform
echo canceller automatically on most modern Android devices, but a
small set of older / OEM-customised devices only attach AEC under
MODE_IN_COMMUNICATION — which an audio-room app deliberately
avoids. Attaching the standalone audiofx effects to the
AudioRecord's session id covers those devices without rerouting
through the call audio path. All three are best-effort and a no-op
on devices where the source already engages them.
#4 Real audio focus handling. The previous OnAudioFocusChangeListener
was a no-op based on the assumption that the OS would auto-duck
us; it doesn't (CONTENT_TYPE_SPEECH streams aren't auto-ducked).
Inbound phone calls were mixing on top of room audio.
- New `NestAudioFocusBus` (commons) — process-wide enum signal,
decoupled from android.media so commons stays platform-free.
- NestForegroundService translates AUDIOFOCUS_GAIN / LOSS_TRANSIENT*
/ LOSS into the bus enum.
- NestViewModel observes the bus and silences both the listener
playback (effective listen-mute = user OR focus) and the
broadcast mic (effective mic-mute = user OR focus). User-visible
mute states stay the user's choice so a focus regain restores
them automatically.
- The pipeline keeps running while focus is lost (decoder, capture,
network) so unmute is sample-accurate when the call ends.
#5 AudioDeviceCallback observability. Registers a callback in
NestForegroundService that logs Bluetooth / wired / USB headset
attach + detach with device type + name. Doesn't drive playback
decisions — Android's auto-routing handles route swaps — but
makes "audio cut out when I plugged in headphones" reports
correlatable with a concrete event for the first time.
#7 Network-change → fast reconnect. Without this, a Wi-Fi → cellular
handover left the QUIC connection sitting on a now-dead socket
until its PTO fired (~30 s) before the wrapper noticed. Now:
- New `NestNetworkChangeBus` (commons) — collapses bursts of
onLost/onAvailable into a single recycle event.
- NestsListener + NestsSpeaker grow `recycleSession()` (default
no-op); the reconnecting wrappers override to close the inner
session so their orchestrator opens a fresh one.
- NestForegroundService registers a default-network callback;
suppresses the first onAvailable (registration callback)
and only publishes on actual default-network changes.
- NestViewModel observes the bus and calls recycleSession on
both wrappers. The SubscribeHandle re-issuance pump (listener)
and the hot-swap publisher pump (speaker) cut existing
subscriptions / broadcasts onto the new session as soon as
it lands — same paths the JWT-refresh recycle uses.
- Manifest gains ACCESS_NETWORK_STATE for
registerDefaultNetworkCallback.
1. Split AudioPlayer.start() into allocate + beginPlayback to restore
the synchronous DeviceUnavailable error path that the previous
pre-roll fix collapsed.
- AudioPlayer gains `beginPlayback()` (default no-op) so test
fakes / desktop sinks aren't forced to grow a method they
don't need.
- AudioTrackPlayer.start() now allocates the AudioTrack + audio-
priority writer thread but does NOT call AudioTrack.play();
beginPlayback() flips the device into the playing state.
AudioTrack in MODE_STREAM explicitly supports write() before
play() per the platform docs, which is exactly the contract
pre-roll wants.
- NestPlayer.play() now calls player.start() synchronously
(caller catches DeviceUnavailable + rolls back the slot like
before) and defers beginPlayback() until pre-roll fills.
2. MediaCodecOpusDecoder: drain output + retry input dequeue before
dropping a frame. The prior 10 ms input-buffer dequeue followed
by an unconditional `return ShortArray(0)` turned every transient
stall (thermal throttling, GC pause, output not yet pulled) into
a 20 ms audio gap. Now we drain whatever output is ready —
freeing input slots — then retry input dequeue with a 5 ms
timeout. Only after both misses do we drop the packet, and even
then we return any output samples that the drain produced so
the player doesn't underrun on the back of one tight cycle. The
decoder's drain logic is extracted into a `drainAvailableOutput`
helper so both the pressure-relief path and the post-queue main
drain share it.
3. ReconnectingNestsListener: exponential backoff for the opener-
throws retry path. Replaces the flat 1 000 ms `SUBSCRIBE_RETRY_BACKOFF_MS`
with 250 → 500 → 1 000 ms, reset on first successful subscribe.
The 250 ms floor is well under moq-rs's typical announce-
propagation latency (< 200 ms), so a subscribe-before-announce
miss usually recovers fast enough that the wrapper SharedFlow's
~1.3 s buffer hides the gap entirely.
1. Listener-side pre-roll + bigger playback buffer + audio-priority thread.
- NestPlayer buffers 5 decoded frames (~100 ms) before starting
the AudioPlayer, masking the first-frame underrun that fires
whenever Compose / GC briefly stalls Main.
- AudioTrackPlayer sizes the AudioTrack at max(minBuffer*16, 250 ms)
instead of minBuffer*4 (~80 ms) and writes via a per-instance
audio-priority single-thread executor (Process.THREAD_PRIORITY_AUDIO)
instead of Dispatchers.IO, so WRITE_BLOCKING never contends with
unrelated IO work.
2. MoqLiteSession.subscribe: hoist response typeCode out of the
collect lambda. readVarint advances pos permanently while
readSizePrefixed only rolls back its own length-varint, so a
chunk-split between type and body would re-read the body's size
prefix as the type code on the next chunk and misframe the
response. Mirrors the same fix already in handleInboundBidi.
3. MoqLiteSession.subscribe: register the subscription in the map
BEFORE writing the SUBSCRIBE bytes on the wire. The relay can
open the first group's uni stream before our continuation
re-enters [state] to register; if so, drainOneGroup looked the
id up against an empty map and silently dropped the frame
(~1 frame / 20 ms gap on first attach). Wrap the post-register
writes so a transport-failure unwinds the orphan registration.
4. Hot-swap moq-lite publisher across JWT-refresh boundaries.
- NestMoqLiteBroadcaster: publisher is now @Volatile + supports
swapPublisher(); capture loop snapshots the reference per frame
and resets framesInCurrentGroup on swap.
- MoqLiteNestsSpeaker implements a new internal
HotSwappablePublisherSource interface that exposes
openPublisherForHotSwap(track) without spinning up a broadcaster.
- ReissuingBroadcastHandle keeps a single long-lived broadcaster
across session recycles when the speaker supports hot swap;
legacy IETF / fake speakers fall back to close-then-restart.
- connectReconnectingNestsSpeaker.orchestrator hoists the old-
session close onto a sibling launch so the wrapper can swap
the publisher into the broadcaster on the new session before
the old session's WebTransport drops. Eliminates the previously-
accepted 50–150 ms audible silence at every JWT refresh.
Group rows into Material 3 cards by section, add tonal icon
containers and trailing chevrons, upgrade section header
typography, and visually mark the danger zone with error colors.
After drafting the congestion-control plan we concluded the audio-rooms
workload doesn't actually need CC — speakers push ~8 KB/sec, which
never fills any modern link's capacity. The one real concern that
surfaced — STREAM retransmit wasting bandwidth on stale Opus frames
on lossy uplinks — is much cheaper to fix directly than to bound via
a 14-test CC subsystem.
SendBuffer gains a `bestEffort: Boolean = false` constructor flag.
When true, markLost drops the lost ranges instead of moving them to
the retransmit queue and lets the underlying byte storage compact as
if the bytes had been ACK'd. The FIN flag (if covered) also stays
sent — best-effort skips FIN re-emission too. The peer may end up
with a truncated stream; moq-lite's per-stream timeouts handle that.
Plumbed through QuicStream → QuicConnection.openUniStream(bestEffort)
→ QuicWebTransportSessionState.openUniStream(bestEffort) →
WebTransportSession.openUniStream(bestEffort). Default is false
everywhere, so reliable streams (HTTP/3 control, moq-lite SUBSCRIBE
bidi, etc.) keep RFC 9000 §3.5 semantics.
MoqLiteSession.openGroupStream now passes `bestEffort = true` —
group streams carry a single Opus packet, are real-time, and don't
benefit from retransmit.
Internal cleanup: `removeOverlap`'s `ackedNotLost: Boolean` parameter
became `OverlapAction { ACK, RETRANSMIT, DROP }` so the third best-
effort disposition has a name. Same code paths, same tests, just
clearer at the call site.
CC plan (quic/plans/2026-05-05-congestion-control.md) is updated to
"parked indefinitely" with a note that this commit is the lighter-
weight alternative that addresses the only practical concern. The
plan is preserved as a reference if a future workload justifies CC.
New tests: SendBufferBestEffortTest (6 cases — reliable baseline,
best-effort drops, FIN drop in best-effort mode, partial overlap,
idempotent stale loss, ACK path still works).
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Open plan, not started. Conservative scope:
- NewReno reference algorithm (RFC 9002 §7).
- Bytes-in-flight tracking + send-side gating.
- Persistent-congestion handling.
- ~14 unit + 2 integration tests, ~3-5 days work.
Out of scope (deferred follow-ups): pacing, CUBIC, BBR, ECN.
The Why section is honest that this is "good citizen" work, not
"fix a bug" work — the audio cliff was a stream-id / forwarding
issue, not a rate-control issue, and the retransmit subsystem we
just shipped is what production actually needed. Plan stays open as
the natural next item but should not be prioritised over field
validation of the retransmit work.
Reference: neqo's cc/classic_cc.rs.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Pre-fix `:quic` held Initial AND Handshake encryption-level state
indefinitely once derived. AEAD cipher state, per-level CRYPTO
buffers, and the per-level sent-packet map all stayed alive for the
lifetime of the connection — a real memory leak for long sessions
(audio rooms run for hours).
LevelState.discardKeys() (idempotent):
- Nulls sendProtection / receiveProtection (frees AEAD state).
- Replaces cryptoSend / cryptoReceive with empty instances.
- Replaces ackTracker with an empty instance.
- Clears sentPackets and resets largestAckedPn /
largestAckedSentTimeMs.
- Latches keysDiscarded = true.
Hook locations:
- Initial discard (RFC 9001 §4.9.1, client side): in
QuicConnectionWriter.drainOutbound, after a Handshake-level packet
is built into the outbound datagram. The next drainOutbound MUST
NOT touch the Initial level; any retransmitted Initial from the
peer is silently dropped (receiveProtection == null), which is
correct per the same RFC since the server has also moved up
encryption levels by then.
- Handshake discard (RFC 9001 §4.9.2 + §4.1.2, client side): in
QuicConnectionParser, on receipt of a HANDSHAKE_DONE frame.
Once a level's protection is null, parser-side decrypt at that level
returns null silently (existing receiveProtection == null check) and
writer-side build skips it (existing sendProtection == null check),
so no further code paths needed updating.
New test: KeyDiscardTest (4 cases — Initial keys discarded after
first Handshake packet, Handshake keys still live until
HANDSHAKE_DONE, Handshake keys discarded on HANDSHAKE_DONE,
discardKeys is idempotent).
Listed in the audit-summary deferred-work as item 3
(`No Initial / Handshake key discard`).
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Pre-fix, QuicConnectionParser purged the inbound AckTracker on every
inbound AckFrame using `frame.largestAcknowledged - frame.firstAckRange`
— but that value lives in OUR outbound PN space, while the tracker
holds inbound PNs we received from the peer. The two PN spaces are
unrelated; the bug mostly hid because they grow at similar rates,
but caused range-list bloat over long sessions where traffic is
asymmetric (e.g. listener receives ~50 audio frames/sec while sending
back ~1 ACK/sec).
The correct semantics: purge only when the peer has confirmed receipt
of OUR outbound ACK frame. Now driven by the ACK-of-ACK dispatch.
- RecoveryToken.Ack changed from data object to data class carrying
(level, largestAcked) — the encryption level and the largest inbound
PN our outbound ACK frame covered.
- QuicConnectionWriter populates these fields from the AckFrame at
emit time.
- QuicConnection.onTokensAcked dispatches RecoveryToken.Ack to
levelState(level).ackTracker.purgeBelow(largestAcked + 1).
- The wrong purge in QuicConnectionParser is removed (replaced with a
comment pointing at the new dispatch path).
Listed in the audit-summary deferred-work as item 6
(`AckTracker.purgeBelow threshold semantics`).
New test: AckTrackerPurgeOnAckOfAckTest (4 cases — purge on
ack-of-ack, level routing, partial purge keeps higher PNs, out-of-order
ACKs are safe).
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Update the two affected plan docs now that RFC 9002 retransmit is
shipped on this branch.
quic/plans/2026-05-04-control-frame-retransmit.md:
- Mark plan as shipped 2026-05-05; list the 15 commits that landed it
(plan + 9 steps + 5 follow-ups + perf + audit).
- Document what changed vs the original scope: the deferred follow-ups
(STREAM data, CRYPTO, RESET_STREAM/STOP_SENDING/NEW_CONNECTION_ID)
all shipped on top of the receive-flow-control core.
- Note the binary-search SendBuffer perf optimisation and the
audit-driven first-call-wins fix.
- Update the cap-workaround status: initialMaxStreamsUni is back to
10_000 (not the 1_000_000 mentioned in the original Why).
quic/plans/2026-04-26-quic-stack-status.md:
- Phase F downgraded from "partial" to "done (no CC)" — loss
detection, RTT estimator, PTO, and per-frame retransmit shipped.
- Removed "no STREAM retransmit" / "SendBuffer doesn't retain bytes
until ACK" from the deliberately-don't-do list (now do).
- Added congestion-control as the new deliberately-don't-do entry.
- Crossed out the corresponding deferred-work items; added congestion
control as deferred item #8.
- Listed the new recovery test files in the test inventory.
- Linked to the retransmit plan + implementation log.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Audit follow-up from the prior two commits.
1. resetStream() / stopSending() now no-op on the second call. RFC 9000
§3.5 pins finalSize at first emission; replaying retransmits with a
larger value (because the app enqueued more bytes between two
resetStream calls) would trigger FINAL_SIZE_ERROR on the peer. The
"idempotent: a second call overwrites with the newer error code"
claim was simply wrong. Two new tests lock the contract:
resetStream_secondCallIsNoOp_finalSizeFrozen and
stopSending_secondCallIsNoOp.
2. resetEmitPending / resetAcked / stopSendingEmitPending /
stopSendingAcked are now @Volatile. The public emit APIs are
callable from any coroutine while the writer / loss / ACK
dispatchers read the same fields under QuicConnection.lock; volatile
gives the cross-thread happens-before, and the first-call-wins gate
above eliminates the only multi-writer race (two app threads racing
the writer's clear-after-emit).
3. SendBuffer's class-level KDoc still claimed range arithmetic was
O(N) "swap to TreeMap if profiling flags it" — stale after the
binary-search refactor in 303caa8. Updated to reflect the actual
O(log N + k) cost.
4. The bulk-removal comment in removeOverlap overstated the win
("O(k) per call, single shift of trailing entries"). ArrayDeque
removeAt(i) shifts on every call, so the actual cost is
O(k * (size - end + k)). Toned down — it's still cheap because
k is 1-2 in steady state.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
removeOverlap was O(N) on the in-flight list — every ACK or loss
notification scanned the whole deque. Replaced with a binary-search
firstOverlapIndex helper plus a forward early-exit walk and a backward
bulk-removal pass. addToInFlight likewise binary-searches for the
middle-insert position instead of linear-scanning.
The list is sorted by offset and non-overlapping by construction, so
firstOverlapIndex finds the first entry whose end-offset > target in
O(log N), and the walk terminates as soon as r.offset >= rangeEnd.
Workload today is small (<100 entries per stream), but audio rooms
with many active streams compound the per-ACK cost.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Application code can now call QuicStream.resetStream(errorCode) and
stopSending(errorCode); the next writer drain emits the matching frame
with a RecoveryToken. Loss dispatch re-flags the per-stream emit-pending
bit; ACK dispatch latches resetAcked / stopSendingAcked so stale loss
notifications can't re-emit. NEW_CONNECTION_ID retransmit drains
QuicConnection.pendingNewConnectionId on next writer pass (no public
emit API since :quic doesn't rotate connection IDs, but the wiring is
in place for a future emit path).
Five tests in ResetStopSendingEmitTest mirror neqo's send-stream reset
coverage: emit-and-token, retransmit-on-loss, ack-then-stale-loss-drop,
stop-sending emission, and NEW_CONNECTION_ID drain.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Closes commits D + E of the deferred-follow-ups pass. With
SendBuffer's retain-until-ACK semantics in commit B and the
Crypto / ResetStream / StopSending / NewConnectionId tokens added
in commit A, the writer now records a Crypto token per CRYPTO
frame at each encryption level (Initial, Handshake) so RFC 9002
retransmit recovers lost handshake bytes.
# Writer side
QuicConnectionWriter.collectHandshakeLevelFrames returns a new
HandshakeLevelContents(frames, tokens) pair instead of a bare
frame list. For each emitted CryptoFrame it appends a matching
RecoveryToken.Crypto(level, offset, length).
QuicConnectionWriter.buildLongHeaderFromFrames now also takes the
parallel tokens list, and after encryption records a SentPacket
in the matching LevelState.sentPackets map. Initial-level rebuilds
with padding (RFC 9000 §14.1) call pnSpace.rewindOutboundForRebuild
to reuse the same PN — the second build's map insert overwrites
the prior entry, so retention reflects the final padded packet.
drainOutbound's two callsites updated to pass tokens through.
# ACK / loss dispatch
Already wired in commit A. QuicConnection.onTokensAcked routes
Crypto tokens to LevelState.cryptoSend.markAcked at the matching
level, releasing buffer memory as the contiguous low end is ACK'd.
QuicConnection.onTokensLost routes them to markLost, re-queueing
the bytes for retransmit at the same level.
# RESET_STREAM / STOP_SENDING / NEW_CONNECTION_ID
Same dispatcher-only completion. The pendingResetStream /
pendingStopSending / pendingNewConnectionId maps on QuicConnection
are populated by the loss dispatcher when those token types are
seen. :quic doesn't currently emit any of those frames (no
application code triggers stream reset, connection-ID rotation
isn't wired), so the writer never drains the maps yet —
scaffolding for future emit support. The exhaustive when in
onTokensLost / onTokensAcked is now complete: any future addition
of a new RecoveryToken variant trips the compile-time exhaustive
check, mirroring the test in RecoveryTokenTest.
# Tests added (3, all pass)
CryptoRetransmitTest:
- handshakePacket_carriesCryptoToken_inSentPacket: ClientHello
emission produces an Initial-level SentPacket with a Crypto
token at offset 0 with the expected level.
- cryptoData_lostAndRetransmittedAtSameLevel: simulate loss via
direct dispatch, observe re-queue in cryptoSend, verify next
drain produces a fresh Initial packet replaying the same
offset (RFC 9000 §13.3 idempotent).
- cryptoAck_releasesBufferAtSameLevel: ACK via onTokensAcked,
cryptoSend's readableBytes drops to 0.
# Net result
Lost handshake bytes (ClientHello, EncryptedExtensions, Certificate,
Finished, NewSessionTicket) are now recovered automatically. The
prior 1-second-fixed-PTO placeholder in QuicConnectionDriver
(commit step 7 of the prior plan) becomes meaningfully more useful
— PTO now wakes the writer to retransmit ACTUAL data, not just
emit empty PINGs.
Full :quic test suite + nestsClient moq-lite + amethyst Android
compile all pass.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Closes commit C of the deferred-follow-ups pass. With the
SendBuffer rewrite in commit B (retain-until-ACK with markAcked /
markLost), the connection now wires STREAM frames into the same
RFC 9002 retransmit path that already handles flow-control
extensions.
# Writer side
QuicConnectionWriter.buildApplicationPacket records a
RecoveryToken.Stream(streamId, offset, length, fin) for every
STREAM frame it emits. The token captures the on-wire byte range
plus the FIN bit so retransmit can reproduce the same StreamFrame
on next drain.
# ACK side
New QuicConnection.onTokensAcked() mirrors onTokensLost. The
parser's AckFrame handler iterates the drained packets and routes
each to onTokensAcked, which:
- For Stream tokens: calls SendBuffer.markAcked(offset, length).
The buffer removes the range from in-flight; if the contiguous
low end is now fully ACK'd, flushedFloor advances and storage
shifts forward.
- For Crypto tokens: same shape, applied to the per-level
cryptoSend buffer (commit E will exercise this path for
handshake reliability — Crypto retransmit is wired now but the
writer's CRYPTO emission path doesn't yet record Crypto tokens;
that's commit E).
- For control-frame and Ack tokens: ACK-no-op. The frame already
did its job by reaching the peer; no per-buffer state to
release.
# Loss side
onTokensLost (commit A) already routes Stream tokens to
SendBuffer.markLost. With commit B's real implementation (was a
no-op stub), this now actually re-queues the byte range for
retransmit. The next writer drain pulls from the retransmit queue
before any fresh sends, with the original offset preserved (RFC
9000 §13.3 idempotent retransmit).
# Tests added (3, all pass)
- streamFrame_carriesStreamToken_inSentPacket: writer emits a
Stream token whose fields match the StreamFrame on the wire
- streamData_lostAndRetransmittedOnNextDrain: simulate loss via
direct dispatch, observe re-emit at the same offset in a fresh
SentPacket (different PN)
- streamData_ackedReleasesBuffer: ACK via onTokensAcked,
enqueue more bytes, observe the next send picks up at the
post-ACK offset (proves bytes were released and floor advanced)
Full :quic test suite, nestsClient moq-lite tests, amethyst Android
compile all pass.
Net result: lost STREAM data (e.g. nestsClient bidi control-stream
bytes — moq-lite Subscribe/Announce control messages travel on
QUIC bidi streams) is now recovered automatically. Audio rooms
benefit indirectly: the relay's announce/subscribe path is more
resilient to packet loss.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Foundation for STREAM and CRYPTO retransmit (commits C, D of the
deferred-follow-ups pass). Replaces the prior best-effort mode
where takeChunk released bytes immediately.
# Three logical regions
The buffer covers `[flushedFloor, nextOffset)`. Each byte is in one
of three states:
- In-flight: sent but not yet ACK'd. Sorted-by-offset list.
- Needs retransmit: declared lost; re-sent before any fresh bytes.
FIFO queue.
- Unsent: `[nextSendOffset, nextOffset)`.
# New API
- markAcked(offset, length): removes the range from in-flight,
advances flushedFloor through any contiguous low-end ACKs and
shifts the underlying byte storage forward. Length=0 means
FIN-only ACK; latches finAcked = true.
- markLost(offset, length, fin): removes from in-flight, appends
to retransmit queue. If fin, clears finSent so the FIN gets
re-emitted. Idempotent: stale loss notifications below
flushedFloor are absorbed.
takeChunk priority order:
1. retransmit queue (preserves original offset; same byte data)
2. fresh unsent bytes from [nextSendOffset, nextOffset)
3. FIN-only zero-byte chunk if finPending and everything drained
# Range arithmetic
removeOverlap walks in-flight, computes the three-piece split for
each overlapping range (leftKept, covered, rightKept) and either
drops the covered portion (ACK) or pushes it onto retransmit
(loss). FIN belongs to the rightmost piece, so split at the end of
the original range correctly preserves it.
# Compaction
advanceFlushedFloorIfPossible bumps the floor whenever the lowest
in-flight / retransmit / unsent offset is above it, then
ByteArray.copyInto shifts the data window forward. Memory bounded
by `nextOffset - flushedFloor` rather than growing unboundedly.
# FIN handling
Treated as a virtual byte at offset = nextOffset. finPending arms
takeChunk to attach FIN to the final data chunk; finSent latches
true on emission; markLost(fin=true) clears it for re-emission;
finAcked latches true when the FIN-bearing range is ACK'd.
# Tests added (14, all pass)
- takeChunk_releasesNothingUntilAcked
- markAcked_full_releasesBytes_andDoesNotResend
- markLost_movesBytesToRetransmitQueue_takeChunkReplaysSameOffset
- markLost_partialRange_splitsInFlight
- markAcked_partialRange_splitsInFlight
- retransmitDrainsBeforeFreshBytes (priority order)
- maxBytesSplits_acrossRetransmitAndFresh
- fin_carriedOnFinalDataChunk_andRetransmittedOnLoss
- fin_only_emittedAfterDataDrained
- fin_only_lostAndRetransmits
- markAcked_advancesFlushedFloor_releasesMemory
- markAcked_outOfOrder_preventsFloorAdvance
- markLost_belowFlushedFloor_isNoop (defensive)
- finAcked_latchesTrueOnce
- readableBytes_reflectsRetransmitPlusFresh
- multipleSendsWithinSingleEnqueue_acksIndependently
Existing :quic tests (handshake, flow control, frame routing) +
nestsClient moq-lite + amethyst Android compile all pass.
SendBufferConcurrencyTest unchanged — the synchronized-on-this
discipline carries over from the prior implementation.
Wires nothing yet — commits C/D will route Stream/Crypto tokens
through markLost. The dispatcher in QuicConnection.onTokensLost
already calls markLost (added in commit A as a no-op stub); now
those calls actually do work.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Token-shape commit (commit A of the deferred-follow-ups pass that
extends RFC 9002 retransmit beyond the receive-side flow-control
extensions originally shipped at commit 9e6fa3d).
New variants on RecoveryToken sealed class:
- Stream(streamId, offset, length, fin): STREAM data range we
sent. On loss the dispatcher will re-queue the range on the
stream's SendBuffer (commits C, D wire that).
- Crypto(level, offset, length): CRYPTO bytes at a specific
encryption level. RFC 9000 §17.2.5 + §13.3: handshake bytes
are reliable; lost CRYPTO retransmits at the same encryption
level. Commit E wires the per-level dispatch.
- ResetStream(streamId, errorCode, finalSize),
StopSending(streamId, errorCode),
NewConnectionId(seq, retirePriorTo, cid, statelessResetToken):
reliable per RFC 9000 §13.3, scaffolding-only since :quic
doesn't currently emit any of these. The pendingResetStream /
pendingStopSending / pendingNewConnectionId maps on
QuicConnection are populated by the dispatcher but not yet
drained by any writer code path.
QuicConnection.onTokensLost extended with the new variants:
- Stream/Crypto: route to the matching SendBuffer.markLost
(currently a no-op — see SendBuffer.markLost kdoc; commit C
replaces the stub with the retain-until-ACK rewrite).
- ResetStream/StopSending/NewConnectionId: write to the
pending* maps; future emit code drains them.
NewConnectionId data-class needs explicit equals/hashCode because
its ByteArray fields would otherwise compare by identity (Kotlin
data-class auto-equals limitation). Tested.
Tests added (4):
- whenDispatch_isExhaustive extended with all 10 variants;
catches at compile time if a new variant is added without
updating the dispatcher
- newConnectionId_arrayEqualityIsByContent
- stream_equalityByValue
- crypto_equalityByValue
SendBuffer gains placeholder markLost(offset, length, fin) and
markAcked(offset, length) methods — both no-ops today. Their
signatures are stable so the dispatcher wiring lands once now and
doesn't need re-touching when commit C makes them functional.
Full :quic test suite passes.
https://claude.ai/code/session_01PYYez8a6sjiakyjAxsfCEQ
Vitor Pamplona