diag(quic-interop): periodic 250ms qlog flush so SIGKILL doesn't strand traces
Pre-fix QlogWriter only flushed in close(); the 60s runner timeout
SIGKILLs the JVM before runTransferTest reaches its qlogWriter?.close().
On every failed quic-go transferloss, the trace ended at exactly 32768
bytes — 4 × 8KB BufferedWriter blocks — masking ~50 seconds of
late-connection behavior. Made every interop debugging session start
with "is this a connection wedge or a qlog wedge?".
Per-event flush was the original shape and was removed in 99a1a91de
because it caused multi-ms stalls on macOS Docker virtualized
filesystems (broke handshakes mid-flight). 250 ms is the compromise:
cheap enough to not stall the send path, fine-grained enough to
capture per-PTO behavior under heavy loss.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -69,6 +69,17 @@ class QlogWriter(
|
|||||||
private val lock = ReentrantLock()
|
private val lock = ReentrantLock()
|
||||||
private val startMillis: Long = nowMillis()
|
private val startMillis: Long = nowMillis()
|
||||||
|
|
||||||
|
/** Last wall-clock millis we flushed at. Periodic flushing avoids the
|
||||||
|
* "qlog truncates at exactly the BufferedWriter buffer size when the
|
||||||
|
* JVM is hard-killed at runner timeout" trap that hides the test's
|
||||||
|
* late behavior. The interval is generous enough to keep the macOS
|
||||||
|
* Docker filesystem virtualization overhead off the hot send path
|
||||||
|
* (per-event flush there was multi-ms and broke handshakes — see
|
||||||
|
* commit 99a1a91de). 250 ms keeps us within ~one cwnd of the
|
||||||
|
* failure under loss while still being far cheaper than per-event. */
|
||||||
|
@Volatile
|
||||||
|
private var lastFlushMillis: Long = startMillis
|
||||||
|
|
||||||
init {
|
init {
|
||||||
// qlog 0.3 JSON-SEQ header. qvis tolerates both `qlog_format`
|
// qlog 0.3 JSON-SEQ header. qvis tolerates both `qlog_format`
|
||||||
// values "JSON-SEQ" and "NDJSON"; we use JSON-SEQ to match the
|
// values "JSON-SEQ" and "NDJSON"; we use JSON-SEQ to match the
|
||||||
@@ -301,16 +312,22 @@ class QlogWriter(
|
|||||||
try {
|
try {
|
||||||
writer.write(line)
|
writer.write(line)
|
||||||
writer.write("\n")
|
writer.write("\n")
|
||||||
// Deliberately NOT flushing per event: this method runs
|
// Periodic flush so a runner-timeout SIGKILL doesn't leave
|
||||||
// inside conn.lock.withLock { drainOutbound(...) } on the
|
// the last seconds of the trace stranded in the JVM buffer.
|
||||||
// hot send path. On macOS Docker Desktop the filesystem
|
// Pre-fix this method never flushed (relied on close()), and
|
||||||
// is virtualized and per-event flush is multi-millisecond.
|
// a 60 s test ended at exactly 32768 bytes = 4 × 8 KB
|
||||||
// Every flush stalls the connection lock, blocks the
|
// BufferedWriter blocks — masking 50 s of late-connection
|
||||||
// receive loop, and the connection silently dies
|
// behavior and turning every interop debug session into
|
||||||
// mid-handshake. close() does the only flush we need
|
// "did the connection wedge or did the qlog?". Per-event
|
||||||
// for normal completion. A hard-killed JVM loses recent
|
// flush was the prior shape and was multi-ms on macOS
|
||||||
// events but that's an acceptable trade — the alternative
|
// Docker virtualized filesystems (commit 99a1a91de). 250 ms
|
||||||
// is the connection never completing in the first place.
|
// is the compromise: cheap enough to not stall the send
|
||||||
|
// path, fine-grained enough to capture per-PTO behavior.
|
||||||
|
val now = nowMillis()
|
||||||
|
if (now - lastFlushMillis >= FLUSH_INTERVAL_MILLIS) {
|
||||||
|
writer.flush()
|
||||||
|
lastFlushMillis = now
|
||||||
|
}
|
||||||
} catch (_: java.io.IOException) {
|
} catch (_: java.io.IOException) {
|
||||||
// Stream closed under us. Latch closed so subsequent emits
|
// Stream closed under us. Latch closed so subsequent emits
|
||||||
// skip the lock and return immediately.
|
// skip the lock and return immediately.
|
||||||
@@ -320,6 +337,8 @@ class QlogWriter(
|
|||||||
}
|
}
|
||||||
|
|
||||||
companion object {
|
companion object {
|
||||||
|
private const val FLUSH_INTERVAL_MILLIS: Long = 250L
|
||||||
|
|
||||||
private val DEFAULT_MAPPER: ObjectMapper = jacksonObjectMapper()
|
private val DEFAULT_MAPPER: ObjectMapper = jacksonObjectMapper()
|
||||||
|
|
||||||
private fun packetTypeFor(level: EncryptionLevel): String =
|
private fun packetTypeFor(level: EncryptionLevel): String =
|
||||||
|
|||||||
Reference in New Issue
Block a user