Three of the bugs that the SQLite review surfaced also live in the
filesystem-backed event store. Bringing both stores back to parity:
- NIP-01 lexical-id tiebreaker (FsSlots, FsEventStore): when two
replaceables / addressables share `createdAt`, the lexically smaller
id wins. The previous `existing.createdAt >= incoming.createdAt`
check rejected equal-timestamp inserts unconditionally — which
blocks the legitimate winner whenever it arrived second.
- delete(Filter()) safe-by-default (FsEventStore): an empty filter
used to enumerate every event and delete each one, so a stray
`delete(Filter())` would wipe the entire on-disk store. Now both
the single-filter and list-of-filters overloads short-circuit when
every filter is empty, matching the SQLiteEventStore contract.
- NIP-09 author check on tombstone install (FsEventStore,
FsTombstones): SQLite's `reject_deleted_events` trigger checks
`event_tags.pubkey_hash = NEW.pubkey_owner_hash`, so a stranger's
kind-5 with an `e`/`a` tag pointing at someone else's event must
not block them from re-publishing. The FS store used to install
the tombstone unconditionally and then read it back without an
author check.
- id tombstones still install (so they can fire when the deletion
arrives before its target), but `idTombstoneOwnerPubKey` is now
compared against the candidate event's owner pubkey at insert
time. GiftWrap parity preserved via FsIndexer.ownerPubKey, which
returns the recipient like the SQLite `pubkey_owner_hash`.
- addr tombstones are now only installed when
`addr.pubKeyHex == deletion.pubKey`, since the address itself
carries the owner identity.
Tests:
- FsSlotsTest: replaces "equal timestamp replaceable is rejected"
(which pinned the buggy behaviour) with two tests covering the
lexical-id tiebreaker in both insertion orders.
- FsParityTest: new same-`createdAt` tiebreaker tests for both
replaceable and addressable kinds, asserting FS and SQLite agree.
- FsDeletionTest:
- inverts "deletion by non-author does not cascade but still
installs id tombstone" — the legitimate owner must be able to
re-insert after the stranger's deletion.
- new test that a stranger's `a`-tag deletion does not block
the legitimate addressable owner from publishing a new version.
- FsEventStoreTest: new `delete with empty filter is safe` test
matching the SQLite-side contract added in batch A.
https://claude.ai/code/session_01X163Nr31vGkvAXoJ3JgMov
idx/tag/<name>/ used to bucket every tag value under a 16-hex Murmur
hash, so `ls idx/tag/p/` showed opaque hex even though pubkey values
are perfectly safe filenames. Now the directory name is:
- the raw tag value, when it's filesystem-safe across Linux/macOS/
Windows: 1..180 bytes, printable ASCII (0x21..0x7e), none of
`/ \ : * ? " < > |`, no leading dot or `_h_`, no trailing dot/space;
- otherwise `_h_<hashHex(murmur)>`. The `_h_` sentinel can never
collide with a raw value (raw values are forbidden from starting
with it), so both forms safely live in the same parent dir.
Common cases keep their raw form and become directly inspectable:
ls idx/tag/p/<your_pubkey>/ — every event that p-tagged you
ls idx/tag/e/<event_id>/ — replies / reactions to an event
ls idx/tag/t/nostr/ — every kind-1 with #nostr
ls idx/tag/k/30023/ — k-tag pointers to articles
ls idx/tag/g/drt3n/ — geohash mentions
Routes through the _h_ bucket: emojis, URLs (slashes), `a`-tags
(colons), free-form `alt` text, anything ≥ 180 bytes, anything that
breaks Windows reserved-name rules. The hash is still seed-salted
Murmur64 (parity with the previous bucket), so collisions are caught
by FilterMatcher post-filter just like before.
Writer (FsIndexer.pathsFor) and reader (FsQueryPlanner.firstTagKey)
both route through FsLayout.tagValueDirName, so they always agree.
Tests: 5 new in FsQueryTest covering raw ASCII tags landing under
named dirs, p-tag pubkeys keeping their raw form, emoji and URL tags
falling back to _h_, and round-trip queries hitting both buckets
correctly. 122 fs tests green.
No migration: existing stores must `amy store scrub` once after the
upgrade — old purely-hashed tag dirs become unreachable, and scrub
rebuilds idx/ from canonicals using the new naming.
Users actually look at <data-dir>/events-store/ files (cat / jq / git
diff), so the CLI now writes them with the InliningTagArrayPrettyPrinter
that quartz already had configured but never invoked. Each event is
indented with 2 spaces, but every tag array stays on a single line —
nice trade-off between human-readable and not-too-tall:
{
"id": "...",
"pubkey": "...",
"created_at": 1700000000,
"kind": 1,
"tags": [
["t","nostr"],
["e","abc...a"],
["alt","quick brown fox"]
],
"content": "hi there",
"sig": "..."
}
Stored bytes are not the canonical NIP-01 form — but verification
re-canonicalises via EventHasher anyway, so format is purely a UX
choice. Compact stays the default for any caller that doesn't opt
in (Android keeps SQLite, generic FsEventStore embedders keep
compact).
- JacksonMapper.toJsonPretty(event): new entry point that uses
writerWithDefaultPrettyPrinter() with the existing inlining printer.
- FsEventStore now takes an `eventToJson: (Event) -> String` callback,
default = Event::toJson (compact). Used in insert.
- Context wires in JacksonMapper::toJsonPretty.
2 new tests in FsEventToJsonTest pin both formats (compact stays
single-line; pretty round-trips). 117 fs tests green.
`profileOf(pk)` and `relaysOf(pk)` are the two hottest read patterns
in any Nostr client, both expressed as `Filter(authors=[pk],
kinds=[0|10002])`. Before this change every such query walked
`idx/kind/<k>/`, sorted the listing, and probed candidates — O(N)
in the kind's total event count even though we always wanted exactly
one event.
The planner now intercepts before the directory walk: when a filter
pins us to replaceable / addressable kinds with `authors` (and
`d`-tag for addressables), we read the slot file directly. One
`Files.exists()` + one `readString()` per (kind, author[, d]) triple,
no listing, no sort.
Falls through to the generic walk for anything that doesn't fit
(non-uniqueness kinds, missing d-tag, search clause, etc.). Two
new tests in FsSlotsTest assert the shortcut serves correctly
even when `idx/` has been externally wiped — proving the
shortcut isn't accidentally relying on the index.
115 fs tests green.
The clock injection existed only for NIP-40 expiration tests. A public
constructor parameter that ~every caller ignores is API clutter, so
move it to a subclass seam:
- FsEventStore is now `open class`; no more `clock: () -> Long` param.
- `protected open fun now(): Long = TimeUtils.now()` is the override
point. Production always takes the default; tests subclass.
- FsExpirationTest gains a private `ClockedStore(root, source)` that
overrides `now()`. Semantics unchanged, all 113 fs tests still green.
Public `FsEventStore` ctor is now `(root, indexingStrategy, relay)` —
no behavioural-drift surface that callers have to learn.
Drives both EventStore (SQLite reference) and FsEventStore with
identical event streams and asserts query() result sets match. SQLite
throws on blocked inserts (NIP-09 tombstones, NIP-62 vanish, NIP-40
expiration) while the FS store silently skips — both are observable
"event not persisted", so insertBoth() catches SQLite throws and
parity is judged on the post-insert state.
Coverage: id lookup, kind+author, since/until/limit, single-letter
tag query (OR within key), replaceable winner, replaceable older
rejected, addressable d-tag dedup, deletion by id (incl. block-
reinsert), deletion by address (incl. cutoff semantics), expiration
sweep, NIP-50 single-token search (sticking to ASCII where SQLite's
unicode61 tokenizer agrees with our port), count, multi-filter
union, delete by filter, mixed kitchen-sink scenario.
113 fs tests now green (97 unit + 16 parity).
- FsLockManager: cross-process exclusive flock(.lock) with per-thread
re-entry. withWriteLock { body } acquires once on a fresh thread
and reuses on nested calls — so transaction { insert(); insert() }
doesn't self-deadlock.
- FsEventStore: insert / delete / delete(filter) / delete(filters) /
delete(id) / deleteExpiredEvents / transaction now run under
withWriteLock. The `*Locked` helpers expose the lock-free body for
re-entrant callers (transaction body, vanish/deletion cascades,
expiration sweep). close() releases the lock channel.
- scrub(): wipes idx/ and rebuilds every entry from the canonical
events. Slots, tombstones and seed are left alone — slots can pin
data the canonical pass doesn't see, and tombstone removal is a
deliberate "un-forget" per the design plan.
- compact(): drops dangling idx/ entries whose canonical no longer
exists. Cheap — only touches idx/, never opens a JSON.
Tests: 11 new in FsMaintenanceTest covering lock file presence,
transaction commit / propagated exception with kept-prior-events
semantics, re-entrant lock from inside a transaction, scrub
rebuilding idx + FTS after a manual wipe, scrub preserving the
replaceable slot, compact dropping dangling and leaving valid alone,
close idempotence + reopen, and two-thread concurrent insert
serialisation. 97 fs tests green.
Adds idx/fts/<token>/<ts>-<id> hardlinks and an FTS-driven query path.
- FsSearchTokenizer: lowercase + Unicode-aware split on non letter-or-
digit, matching SQLite FTS5's unicode61 default closely enough that
the same call indexes content and parses queries (any drift cancels).
Tokens capped at 100 chars to keep filenames under FS limits.
- FsLayout: idxFts + ftsEntry / ftsTokenDir helpers; skeleton dir.
- FsIndexer.pathsFor: when event implements SearchableEvent, emits one
hardlink per unique tokenised word — so insert/delete maintenance
rides the existing link/unlink path. Eviction (replaceable swap),
NIP-09 cascade and NIP-62 vanish all clean up FTS for free.
- FsQueryPlanner: when filter.search is non-blank, drives by FTS.
Tokenises the query, walks each idx/fts/<token>/ listing into a
HashMap<id, ts>, and intersects smallest-first (AND across tokens —
matching SQLite FTS5 default MATCH semantics). Output sorted by
createdAt DESC. Other Filter fields (kinds, authors, tags, since /
until) still apply via Filter.match post-filter.
Tests: 16 new in FsSearchTest covering tokenizer (whitespace, case,
unicode, punctuation, empty), index maintenance (entries created,
non-searchable kinds skipped, delete unlinks), and query semantics
(single token, AND of tokens, ordering, limit, kind/author compose,
no-match, blank string ignored, reopen). 86 fs tests green.
Adds tombstones/vanish/<owner_hex>.json hardlinks (one per owner,
strongest cutoff wins) plus the cascade and block-future-insert
semantics from SQLite's RightToVanishModule.
- FsLayout: vanishTombstonePath helper + tombstones/vanish/ skeleton.
- FsTombstones: vanishCutoff / installVanish / clearVanish — install
uses atomic rename-with-REPLACE_EXISTING and only proceeds when the
new kind-62's createdAt is strictly greater than any existing tomb.
- FsEventStore:
- constructor now takes optional relay: NormalizedRelayUrl? to scope
NIP-62 cascades (matches SQLiteEventStore's relay arg).
- isBlockedByTombstone now also rejects events whose owner has an
active vanish with createdAt >= event.createdAt — owner is the
recipient for GiftWrap, matching pubkey_owner_hash semantics.
- processVanish() runs after canonical write: skips if !shouldVanish-
From(relay), installs the tombstone, then walks idx/owner/<hex>/
and deletes every event with ts < vanish.createdAt. The vanish
event itself survives (its ts equals the cutoff).
Tests: 11 new in FsVanishTest — relay-scoped cascade, different-relay
no-op, vanishFromEverywhere, block re-insert + newer-passes, equal
ts blocked, other authors unaffected, stronger cutoff wins, weaker
ignored, tombstone shares inode with kind-62 canonical, kind-62 stays
queryable. 70 fs tests green.
Adds idx/expires_at/<padded_exp>-<id> hardlinks for events with an
expiration tag, an injectable clock so tests can drive time
deterministically, and the deleteExpiredEvents() sweep.
- FsLayout: idxExpiresAt + expirationEntry path helper.
- FsIndexer.pathsFor: emits the expiration entry whenever
event.expiration() > 0, so insert / delete maintain it alongside
the kind / author / owner / tag indexes.
- FsEventStore: pre-insert guard rejects events with exp <= now
(SQLite parity: trigger uses inclusive <=). Constructor takes a
clock function defaulting to TimeUtils.now(). deleteExpiredEvents
walks idx/expires_at, parses filenames, and deletes anything with
exp < now (strict <, matching SQLite's sweep query).
Tests: 8 new in FsExpirationTest — future expiration accepted +
indexed, already-expired-on-insert rejected, exp==now rejected on
insert but kept by sweep, non-positive exp ignored, sweep removes
canonical + index entries, plain events untouched. 59 fs tests green.
Tombstone files under tombstones/id/<id>.json and tombstones/addr/
<kind>/<pubkey>/<sha256(d)>.json, each a hardlink to the kind-5
event that authored the deletion. One source of truth: the tombstone
IS the deletion event, just indexed by target.
- FsTombstones — installs id tombstones unconditionally, installs
addr tombstones with strongest-cutoff-wins semantics (later kind-5
replaces earlier via atomic rename), and exposes hasIdTombstone /
addrTombstoneCutoff for pre-insert checks.
- FsEventStore.insert — pre-insert guard: id tombstone always blocks;
addr tombstone blocks when event.createdAt <= tomb.createdAt, matching
SQLite's reject_deleted_events trigger. Kind-5 inserts trigger a
cascade: for each e-tag target owned by the deletion author, unlink
indexes + slot + canonical; for each a-tag (same pubkey), evict the
slot winner if its createdAt <= deletion.createdAt. Tombstones are
installed for every target regardless, so future re-inserts are
blocked.
Tests: 11 new in FsDeletionTest — delete-by-id, block-reinsert-by-id,
non-author-deletion still installs tombstone but no cascade, cascade
addressable slot, newer-at-deleted-address passes, older blocked,
equal-timestamp blocked, later kind-5 raises cutoff, earlier kind-5
does not lower, deletion event stays queryable, tombstone shares
inode with kind-5 canonical. 51 fs tests green.
Brings the file-backed store up to parity with SQLite's ReplaceableModule
and AddressableModule. A slot is a single hardlink that encodes the
UNIQUE(kind, pubkey[, d-tag]) constraint directly in the directory
layout:
replaceable/<kind>/<pubkey>.json (kinds 0, 3, 10000-19999)
addressable/<kind>/<pubkey>/<sha256(dTag)>.json (kinds 30000-39999)
FsSlots handles the full lifecycle: pre-insert guard (reject if newer or
equal exists), atomic rename-with-REPLACE_EXISTING install, and eviction
of the old winner's canonical + index hardlinks. Because the slot is a
hardlink the event data survives external canonical deletion, matching
the "files come and go" contract in the design plan.
delete(id) also clears the slot when the deleted event is the current
winner, so no orphan slot files linger.
Tests: 14 new in FsSlotsTest — newer wins / older rejected / equal
rejected / eviction unlinks old indexes / empty d-tag / canonical-
deletion survives via hardlink / delete-clears-slot / non-replaceable
events never touch the slot dirs. 40 fs tests pass.
Hardlink indexes under idx/ and a minimal query planner bring the
file-backed store up to parity with SQLite on filtered lookups.
- FsLayout — path helpers, .seed file (8 random bytes, salts all hashes),
entry filename format <zero-padded ts>-<id>.
- FsIndexer — on insert creates hardlinks at idx/kind/<k>/, idx/author/
<pk>/, idx/owner/<owner_hex>/, idx/tag/<name>/<hash_hex>/. Owner hash
matches SQLite's pubkey_owner_hash (recipient for GiftWrap). Honours
DefaultIndexingStrategy: single-letter tag names only. On delete
unlinks every known path so the inode can be reclaimed.
- FsQueryPlanner — picks a driver (ids / first tag / kinds / authors /
all kinds) and yields candidates sorted by createdAt DESC. Final
predicate check runs through Filter.match so any driver is
correctness-safe.
- FsEventStore — sets mtime to event.createdAt on write, runs queries
through the planner with post-filter, applies limit, dedupes by id
across multi-filter unions, and unlinks indexes on delete.
Tests: 16 new in FsQueryTest covering order, limit, author / kind /
tag drivers, tag OR within a key, tagsAll AND across keys, non-
single-letter-tag behaviour, since/until, count, index hardlink
maintenance, and reopen persistence. All 26 fs tests green.
First slice of the file-backed IEventStore planned in
cli/plans/2026-04-24-file-event-store-*. Each event is stored as
events/<aa>/<bb>/<id>.json with atomic tmp+rename writes. Ephemeral
kinds are dropped. Duplicate inserts are no-ops (id-level uniqueness).
Staging leftovers are swept on open.
Only id-based query and delete are wired; the query planner, indexes,
replaceable/addressable slots, tombstones, vanish, expiration sweep,
FTS, and transactions land in later steps.
Tests: 10 new tests under quartz jvmTest, all passing.
Correctness (C):
- scalar_mul: rewrite with a loop-driven fold so the 512-bit product is
fully reduced regardless of the pre-existing third-fold carry-drop bug;
also fixes the portable (!HAVE_INT128) fallback which was returning
(a*b) mod 2^256 instead of (a*b) mod n.
- fe_mul / reduce_wide: loop the final carry fold in a while(carry) rather
than a single if(carry), so a secondary carry-out is never silently
dropped for adversarial or deeply lazy-reduced inputs.
- scalar_add: reuse the precomputed SCALAR_NC constant instead of
recomputing n's two's-complement arithmetically each call.
- fe_negate: remove the data-dependent early-return for a == 0; compute
P - a unconditionally and fold P back to 0 with a final fe_normalize,
matching the Kotlin FieldP.neg path and dropping a branch.
Performance (C):
- Add a dedicated 10-mul fe_sqr_inline using __int128 (4 diagonal + 6
doubled cross products, three-pass structure mirroring Kotlin
U256.sqrWide). The previous fe_sqr delegated to fe_mul(a, a) using all
16 schoolbook products; the new path saves ~37% of the multiplications
at every squaring, and doublePoint/addPoints do ~9 sqrs each in the hot
Jacobian loop. Col-3 mixes two products so the accumulator is split
explicitly to avoid a uint128 overflow; the bug was caught by the C
benchmark's self-test.
- Enable LTO (CMAKE_INTERPROCEDURAL_OPTIMIZATION) when the toolchain
supports it, recovering cross-TU inlining of fe_mul/fe_sqr into point.c
and schnorr.c.
- jni_bridge: replace the pinning GetByteArrayElements path (which blocks
GC compaction) with a stack-or-heap copy_msg_bytes helper that uses
GetByteArrayRegion. Short messages (32 B event digests, the common case
for Nostr) hit a 512 B on-stack buffer.
Correctness (Kotlin):
- FieldP.neg: remove the early-return on zero for the same reasons as the
C side, with a trailing reduceSelf(out) to collapse the P result back
to 0 when the input was zero.
- Secp256k1.signSchnorrInternal / privKeyTweakAdd: switch the crypto-
edge-case failure modes from generic require() to check() with clear
messages, documenting them as invariants rather than argument errors
and matching the C side's return-code semantics.
Tests & benchmarks:
- Add Secp256k1CrossValidationTest (jvmTest) that byte-for-byte compares
Kotlin, ACINQ, and the custom C implementation across pubkey creation,
Schnorr signing (incl. variable message lengths on the Kotlin side),
privKeyTweakAdd, and x-only ECDH. This is the strongest parity check we
can run without a third reference, and it's deterministic for
reproducibility (fixed LCG seed).
- Add adversarial FieldPTest cases that chain lazy adds into mul/sqr/inv
to exercise the new fe_mul fold loop and the dedicated fe_sqr path.
- Fix pre-existing FieldPTest/GlvTest failures (addNearP, addNegIsZero,
halfOfOdd, invMulIsOne, invOfTwo, reduceWideWithMaxValues, betaCubedIsOne)
that were asserting on raw limbs of lazy-reduced values; they now
reduceSelf before comparing, consistent with the rest of the suite.
- Secp256k1Benchmark: document the intentional apples-to-apples
asymmetries (signSchnorrWithPubKey vs signSchnorr, ecdhXOnly vs
pubKeyTweakMul, privKeyTweakAdd's copyOf() penalty) and add a
taggedHash benchmark since NIP-44 leans heavily on it.
All 188 secp256k1 Kotlin tests pass; the C library builds cleanly with
LTO enabled and the secp256k1_bench self-verification succeeds.
https://claude.ai/code/session_01KExJURZATpL59ZKXP6AVP6
Kotlin does not allow crossinline parameters to be nullable. The cOp
parameter in benchTriple needs to be nullable (null when C library is
not available), so use noinline instead.
https://claude.ai/code/session_011KVZhDcV2G7idNWEBz12GY
Add a complete C implementation of secp256k1 elliptic curve operations
alongside the existing Kotlin implementation, enabling direct comparison
and extraction of maximum performance from each platform (ARM64, x86_64).
C Implementation (quartz/src/main/c/secp256k1/):
- field.h/c: 5x52-bit limb field arithmetic with __int128 support and
lazy reduction (12-bit headroom per limb vs Kotlin's fully-packed 4x64)
- scalar.h/c: Scalar mod n arithmetic, GLV decomposition, wNAF encoding
- point.h/c: Jacobian point operations (3M+4S double, 8M+3S mixed add),
GLV+wNAF scalar multiplication, Strauss/Shamir dual scalar multiply,
Montgomery batch-to-affine, precomputed G tables (wNAF-12)
- schnorr.c: BIP-340 Schnorr sign/verify/verifyFast/verifyBatch with
pubkey decompression cache and precomputed tag hash prefixes
- sha256.c: Self-contained SHA-256 for BIP-340 tagged hashes
- secp256k1_c.h: Public API matching the Kotlin Secp256k1 object
- jni_bridge.c: JNI bridge for JVM/Android integration
- benchmark.c: Standalone C benchmark (cmake build)
- CMakeLists.txt: Build system with ARM64/x86_64 optimization flags
Kotlin Integration:
- Secp256k1InstanceC: expect/actual wrapper (commonMain/jvmMain/androidMain/nativeMain)
- Secp256k1C: JVM JNI binding class
- Secp256k1TripleBenchmark: Three-way JVM benchmark (ACINQ vs Kotlin vs Custom C)
- Secp256k1CBenchmark: Android benchmark for the C implementation
Current status: sign works correctly (verified against BIP-340 test vectors),
verify path needs ecmult_double_g debugging (GLV wNAF-12 table issue). The
comb table for ecmult_gen also needs fixing (currently falls back to GLV+wNAF).
Field arithmetic is fully verified: 5x52 limbs with R=0x1000003D10 fold.
https://claude.ai/code/session_011KVZhDcV2G7idNWEBz12GY
Full migration of the secp256k1 library from LongArray(4)/LongArray(8)
to Fe4/Wide8 struct types with @JvmField named Long fields. This
eliminates all array bounds checks from the hot path.
Files migrated (13 source + 7 test + 2 benchmark):
- U256.kt, FieldP.kt, ScalarN.kt, Glv.kt, ECPoint.kt
- FieldMulPlatform.kt (expect + 3 actuals), FieldMulFused.kt
- PointTypes.kt (MutablePoint, AffinePoint, PointScratch)
- KeyCodec.kt, Secp256k1.kt
- All test files and benchmarks
Bytecode impact:
Before: 464 laload/lastore (bounds-checked) in core arithmetic
After: 0 laload/lastore, all getfield/putfield (no checks)
The public API (Secp256k1 object) is unchanged - it still accepts
and returns ByteArray. Fe4 conversion happens at the API boundary
via U256.fromBytes()/U256.toBytes().
All secp256k1 unit tests pass on JVM.
https://claude.ai/code/session_01Sxi6Gpxbstuj3Y8TBY7XrU
BIP-340 verification checks both R.x == r AND R.y is even. The y-parity
check requires a full field inversion (~270 field ops, ~14% of verify).
verifySchnorrFast skips the y-parity check, verifying only the
x-coordinate in Jacobian coordinates (2 field ops, no inversion).
WHY THIS IS SAFE FOR NOSTR:
For a given x on secp256k1, there are exactly 2 points: (x, y_even) and
(x, y_odd). A signature producing the correct x but wrong y-parity would
require solving the discrete log — equivalent to forging the signature.
The y-parity check is defense-in-depth, not a distinct security boundary.
DO NOT use for Bitcoin/financial protocols — use verifySchnorr for strict
BIP-340 compliance.
JVM benchmark:
verifySchnorrFast: 20,706 ops/s (1.4× vs native C)
verifySchnorr: 18,038 ops/s (1.6× vs native C)
Improvement: ~15% faster
https://claude.ai/code/session_01EMY5RnXb9rnsyU2KbXrSaY
Add verifySchnorrBatch(pub, signatures, messages) that verifies n
signatures from the same public key using scalar and point summation
instead of n individual mulDoubleG calls.
The batch equation exploits linearity of Schnorr signatures:
(Σ sᵢ)·G = (Σ Rᵢ) + (Σ eᵢ)·P
This combines n verifications into:
- n scalar additions for S = Σsᵢ and E = Σeᵢ (trivial)
- n liftX + (n-1) addMixed for R_sum = ΣRᵢ (cheap point additions)
- ONE mulDoubleG(S, P, -E) (the expensive EC operation, done once)
- Check result - R_sum = O (no toAffine needed)
JVM benchmark results (same pubkey, 1000 iterations, 500 warmup):
batch( 4): 4.1x faster than individual (40,121 events/s)
batch( 8): 5.3x faster (49,637 events/s)
batch(16): 5.5x faster (49,815 events/s)
batch(32): 5.9x faster (51,272 events/s)
This fits the Nostr pattern perfectly: when loading a profile or
connecting to a relay, many events from the same followed author
arrive together and can be batch-verified.
If the batch fails (returns false), the caller falls back to
individual verification to identify which signature(s) are invalid.
Security: by linearity, individual errors cannot cancel without
solving the discrete log. For extra hardening, duplicate events
from multiple relays can be verified individually as cross-checks.
https://claude.ai/code/session_017UbWduFi1sLUsgVUMUH2nx
The previous counts (200 warmup / 500 iterations for verify) were too
low for reliable measurements on noisy VMs. Increase to 2000+ warmup
and 3000-50000 iterations depending on operation cost, giving the JIT
time to fully optimize and reducing variance between runs.
https://claude.ai/code/session_017UbWduFi1sLUsgVUMUH2nx
Update KDoc and file headers across Secp256k1.kt, Point.kt, FieldP.kt,
and U256.kt with:
- Accurate benchmark numbers (well-warmed: verify 4.4x, sign 1.5x,
pubCreate 2.2x, ECDH 3.9x, compress 2x FASTER)
- Detailed comparison with C libsecp256k1 architecture choices
- Explanation of why certain C optimizations don't port to JVM:
* Lazy reduction: 4x64 limbs have no headroom (C's 5x52 has 12 bits)
* safegcd: slower on JVM due to 128-bit arithmetic overhead
* WINDOW_G=15: cache pressure from heap-allocated objects (w=12 optimal)
- Document effective-affine technique and batch inversion
- Increase all benchmark warmup/iterations for stable measurements
https://claude.ai/code/session_01BhU63WUe9AhikZxRdw3Lpg
Tested WINDOW_G=15 (matching C libsecp256k1): 5.6x slower than native.
WINDOW_G=12 is faster at 4.3x because the 8192-entry table (1MB) at
w=15 causes cache pressure on JVM — heap-allocated AffinePoint objects
are scattered in memory, unlike C's contiguous compile-time .rodata.
WINDOW_G=12 (1024 entries, ~128KB) fits comfortably in L2 cache.
Increased verify benchmark warmup to 200+500 for stable measurements
(first call builds the lazy table).
https://claude.ai/code/session_01BhU63WUe9AhikZxRdw3Lpg
- Add ECPoint.toAffineX that computes only x = X/Z² (saves 2M vs full
toAffine which also computes Y/Z³)
- Use toAffineX in ecdhXOnly since only the x-coordinate is needed
- Add ecdhXOnly benchmark measuring the actual Nostr ECDH production
path (Secp256k1Instance.pubKeyTweakMulCompact delegates to ecdhXOnly)
- Fix ktlint KDoc-inside-class-body violations
The old benchmark measured pubKeyTweakMul(02||x, key) which pays for:
array allocation, compressed key parsing (sqrt), full toAffine, and
re-serialization. ecdhXOnly avoids the array overhead and serialization.
https://claude.ai/code/session_01BhU63WUe9AhikZxRdw3Lpg
Analysis of the C library's schnorrsig_sign showed three key differences:
1. C takes a pre-computed keypair (no pubkey derivation during signing)
2. C does NOT self-verify the signature after signing
3. C does 1 EC multiplication total; we were doing 3
Changes:
- signSchnorrInternal: extracted core signing logic without pubkey derivation
or self-verification. Does exactly 1 mulG (for R = k·G) matching the C library.
- signSchnorr: convenience overload that derives pubkey then calls internal.
Now ~2x faster since self-verify is removed.
- signSchnorrWithPubKey: fast path accepting a 33-byte compressed pubkey
(includes y-parity in the 02/03 prefix). Skips the pubkey G multiplication
entirely. ~4.8x faster than the previous signSchnorr.
- Secp256k1Instance: added signSchnorrWithPubKey forwarding method.
- Benchmark: added "signSchnorr (cached pk)" test comparing both paths.
The self-verify removal is safe: the BIP-340 test vectors (which include
the exact expected signatures) validate correctness, and the C reference
library does not self-verify either.
Benchmark:
signSchnorr: 1,516 → 2,915 ops/s (1.9x faster, 17x → 9.5x)
signSchnorr (cached pk): N/A → 7,261 ops/s (3.9x vs native — new!)
https://claude.ai/code/session_01BhU63WUe9AhikZxRdw3Lpg
Fixes discovered while porting Namecoin NIP-05 to Primal Android
(PrimalHQ/primal-android-app#934):
1. Race condition: stale intermediate lookups overwrite final result
- Changed map to mapLatest in SearchBarViewModel so that previous
in-flight ElectrumX lookups are cancelled when the query changes
- Without this, typing 'd/testls' character by character causes
stale results from 'd/test', 'd/testl' etc. to arrive after
the correct 'd/testls' result and overwrite it with null
2. CancellationException swallowed in catch block
- mapLatest cancels previous coroutines via CancellationException
- The generic catch was catching it and emitting null, wiping the
valid result. Now re-throws CancellationException
3. Root lookup fails when names object has no underscore key
- extractFromDomainValue tried names[localPart] then names[underscore]
which is the same thing for root lookups
- Now falls back to first available entry for root lookups when
no underscore key exists
- Non-root lookups still fail correctly (no false matches)
4. Global Mutex serializes all lookups
- Single Mutex in ElectrumxClient blocked unrelated lookups to
different servers. A 25s timeout on one server stalled everything
- Now uses per-server mutexes via ConcurrentHashMap
5. customServers set but never read
- NamecoinNameService.setCustomServers() stored the list but the
resolver was constructed with the default serverListProvider
- Now wires customServers through to the resolver
6. resolveLive uses orphaned CoroutineScope
- Now accepts an optional external scope parameter so callers can
tie resolution to their own lifecycle
Adds 2 new tests: root fallback to first entry, non-root no-fallback.
Add censorship-resistant NIP-05 verification using the Namecoin blockchain.
Users can set their nip05 field to a .bit domain (e.g. alice@example.bit)
or direct Namecoin name (d/example, id/alice) and Amethyst will resolve
the pubkey mapping via ElectrumX instead of HTTP.
Resolution uses the standard Electrum protocol (scripthash-based lookups):
- Build canonical name index script matching ElectrumX-NMC indexing
- Query blockchain.scripthash.get_history for the name's tx history
- Parse NAME_UPDATE script from the latest transaction output
- Extract Nostr pubkey from the name's JSON value
Supports both d/ (domain) and id/ (identity) Namecoin namespaces,
simple and extended NIP-05-like value formats with relay hints,
LRU caching with 1h TTL, and self-signed TLS certificates.
New files:
- quartz: ElectrumxClient, NamecoinNameResolver, NamecoinLookupCache
- amethyst: NamecoinNameService, Nip05NamecoinAdapter, NamecoinVerificationDisplay
- docs: namecoin-nip05-design.md
- tests: NamecoinNameResolverTest
Modified:
- Nip05Client: optional namecoinResolver routes .bit to blockchain
- AppModules: wire up resolver
See docs/namecoin-nip05-design.md for full architecture and protocol details.
- Fully converts OpenTimestamp Java codebase to Kotlin, migrating the sync and async HTTP call interfaces to OkHttp and coroutines
- Redesigns parsing of relay commands, messages and filters for performance in Jackson.
- Starts the use of KotlinX Serialization when speed is not a requirement
- Migrates all Jackson field annotations to Kotlin Serialization
- Migrates Regex use in Quarts to Kotlin's Regex class
- Migrates Base64 from Android to Kotlin
- Migrates UUID from Android/Java to Kotlin
- Migrates LRUCache usage from Android/Java to Kotlin collections
- Migrates all String to bytearray conversions to Kotlin methods
- Migrates all System.arraycopy calls to kotlin native ones.
- Separates parsing code from the data classes Companion objects
- Exposes Rfc3986 normalizations to each platform.
- Exposes URI parsing classes to each platform.
- Exposes URL Encoders to each platform.
- Exposes BigDecimal to each platform.
- Exposes the Url Detector to each platform.
- Exposes MacInstances to each platform
- Exposes Diggest instances to each platform.
- Exposes a BitSet to each platform.
- Exposes GZip to each platform.
- Exposes Secp256k1 to each platform.
- Exposes SecureRandom to each platform.
- Exposes Time in seconds to each platform.
- Exposes the LargeCache to each platform.
- Exposes AES CBC and AES GCM encryption/decryption to each platform
- Migrate test assertions to Kotlin Tests
- Exposes Address class to each platform because of the Parceleable
- Creates our own ByteArrayOutputStream.
- Removes Lock features inside the Bloomfilters because we don't need data consistency/
- Migrates UserMetadata parser from Jackson to Kotlin serialization
- Removes the need for Static methods in each tag.
- Adds an event template serializer
- Adds KotlinX Datetime to migrate some of the date-based logs
- Adds support for LibSodium in the JVM platform
- Creates a shared test build for iOS targets
- Fixes several usages of Reflection when serializing classes
- Fixes a bug on loading RelayDB for the HintBloom filter test
- Increases the Bloom filter space to better use hints in the app.
- Removes support for iOS in x86
- Creates a Jackson mapper just for NIP-55, which stays in the Android build only.
- Keeps the event store in the android build as well.
- Removes @Syncronized tags in favor of Mutexes.
- Improved sendAndWaitForResponse NostrClient method to properly account for returns from each relay.
- Removes the need for GlobalScope and async calls in the downloadFirstEvent method.
- Restructures the parser and serialization of the relay messages and commands for performance with Jackson
- Removes the dependency on Jackson's error classes across the codebase.
- Moves the hint to quote tag extension methods to their own packages.
- Speeds up the generation of Bech32 addresses
- Migrates NIP-6 and Blossom uploads to use Kotlin Serialization
Claude