Quartz's RelayUrlNormalizer.isLocalHost strips literal 127.0.0.1 /
localhost / 192.168.* / .local / umbrel out of NIP-17 inbox
(kind:10050) and KeyPackage (kind:10051) relay-list events as a
privacy guard. The marmot harness was binding the loopback relay to
ws://127.0.0.1:8080, so when amy added that URL to its kind:10050 /
kind:10051 events the parser silently dropped it on read — leaving
the harness publishing to Amethyst's PUBLIC default relays
(nos.lol, nostr.mom, …) instead of the local one. Whitenoise (which
only listens on the local socket) never saw amy's KeyPackage,
breaking every Test 01–16 scenario at the first hop.
The DM harness already worked around this by using 127.0.0.2 (still
pure loopback, not on the strip list — see dm-interop-headless.sh:34).
This commit applies the same workaround to the marmot harness:
- Default RELAY_HOST=127.0.0.2 (overridable via env or --host).
- RELAY_URL is now derived from RELAY_HOST + RELAY_PORT.
- New --host flag for parity with the DM harness.
setup.sh's relay config already reads RELAY_HOST when binding
nostr-rs-relay (line 173, address = "${RELAY_HOST:-127.0.0.1}"), so
no other change is needed — the relay binds where amy is trying to
reach it.
The grow animation used independent scaleX/scaleY derived from the source
rect (square avatar) and the image rect (often landscape), so at the start
of the transition the full image was squashed into the avatar's square
footprint before lerping back to its real proportions.
Switches to a uniform scale based on max(src.width/img.width,
src.height/img.height) and centers the image on the tapped rect. The image
now keeps its aspect ratio throughout, covering the source rect in at
least one dimension at progress=0 and growing uniformly to fullscreen.
Also extends the entry guard to require non-zero source rect; without it
a not-yet-measured thumbnail would yield startScale=0 and collapse the
first frame to an invisible point before the animation expanded it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
USAGE.md is now the single home for "how do I use amy" — install,
quick start, examples, command reference, troubleshooting. README
was carrying all of that PLUS the public-contract material; with
USAGE in place it can collapse to just the contract.
README.md (338 → 176 lines):
- Keep: 1-paragraph intro (three audiences), output contract, local
event-store explainer, relay-routing rules, on-disk layout, cross-
references to USAGE/DEVELOPMENT/ROADMAP.
- Drop: install, quick start, command reference table, global flags
table, account-management verbs, troubleshooting (all in USAGE).
DEVELOPMENT.md:
- Architecture diagram updated to reflect new files (Aliases.kt,
UseCommand.kt, ProfileCommands.kt, DmCommands.kt, NotesCommands /
PostCommand / FeedCommand, MarmotResetCommand, StoreCommands,
SecureFileIO, secrets/ subtree).
- "Keep three things in sync" pointers redirected from README's
command table to USAGE.md.
- Testing table loses the duplicate "Interop with other clients" row
(already covered by the harness row above).
- Cross-reference list at the top now mentions USAGE.md.
ROADMAP.md:
- Cross-reference list adds USAGE.md.
- Parity matrix marked ✅ for items shipped on this branch:
notes post + feed (PostCommand/FeedCommand), profile show+edit
(ProfileCommands), DMs (already ✅).
- Reactions split into "✅ in groups, 🆕 elsewhere" since
marmot message react is shipped but outer-event reactions aren't.
- Order-of-operations entries marked ✅ where relevant.
Modeled on nostrcli.sh's docs page: quick start, seven worked examples
(post a note, send a DM, read a thread, view a profile, MLS group
round-trip, account switching, add a relay), categorised command
reference, output-modes section, multi-account workflow, agent/script
recipes, troubleshooting.
README.md and DEVELOPMENT.md are referenced from here; they stay
focused on the public contract and the extension rules.
The marmot harness surfaced the bug on first run: amy stripped
`marmot group create --name "Interop-02"` thinking the global
account selector ran into the group's display-name flag. Result:
amy resolved the "Interop-02" account, found no identity.json, and
errored — no group ever got created.
Renames the global account-selector flag to `--account`. The per-
subcommand `--name` flags (`marmot group create --name "Demo"`,
`profile edit --name "Alice"`, `marmot await group --name X`) are
untouched — they're free of the global parser now that it doesn't
claim the same name.
Sweep:
- `Main.kt`: GlobalFlag.NAME → ACCOUNT, long "--account".
- `Config.kt`: DataDir.resolve param renamed nameFlag → accountFlag;
every error message points the user at --account.
- `UseCommand.kt`: error hint says `amy --account NAME init`.
- All test wrappers + direct $AMY_BIN calls under cli/tests/ swap
the global `--name X` for `--account X` (subcommand --name kept
exactly where it appeared).
- README + DEVELOPMENT updated.
There is no installed base to protect, so the self-contained
`--data-dir P` escape hatch goes away entirely. amy now has exactly
one layout — the production multi-account one — and tests exercise
that same code path. Drops one global flag, one DataDir construction
mode, and the "tests use a different code path than users" footgun.
Layout (unchanged from the previous commits — just the only mode now):
~/.amy/
├── current # `amy use NAME` marker
├── shared/
│ └── events-store/ # FsEventStore, one per machine
├── alice/
│ ├── identity.json
│ ├── state.json
│ ├── aliases.json # {"alice": "<own npub>"} after init
│ └── marmot/
└── bob/ ...
`DataDir.DEFAULT_ROOT` reads `$HOME` directly (falling back to
`user.home`) because JDK 21 resolves `user.home` via getpwuid and
ignores `$HOME` — which would have broken the standard CLI test
isolation pattern of `HOME=/tmp/foo amy …` (the same convention
git, gpg, npm follow).
Test sweep:
- `cli/tests/headless/helpers.sh`, `cli/tests/dm/setup.sh`,
`cli/tests/cache/cache-headless.sh` wrappers all switch to
`HOME=$STATE_DIR amy --name X …`.
- `ensure_identity_for` drops its `dir` parameter; the function and
every harness call site go through `--name` only.
- `A_DIR`/`B_DIR`/`D_DIR` get repointed at `$STATE_DIR/.amy/X`. The
one consumer (`cache-headless.sh`'s T6 `relays.json` check) still
works since it's just a path probe.
- `cli/tests/dm/tests-dm.sh` ghost identities get their own
short-lived `$HOME` so they don't pollute the main test root.
Cache-test T4 inverts: pre-shared-store, "B has not seen A → first
profile show is a relay miss, second is a cache hit" tested
per-account caching. With one shared events-store, B's first lookup
of A is already a cache hit because A wrote kind:0 there during
bootstrap. T4 now asserts that — drops the stale "second lookup hits
cache" half. T7 (no-identity maintenance verbs) gets a fresh fake
`$HOME` plus a throwaway `--name` so the empty store has no inherited
events.
Docs (README + DEVELOPMENT) rewritten to match — quick-start now uses
`amy --name alice create`, the on-disk-layout section shows the new
tree, and the global-flags table replaces `--data-dir` with `--name`
plus the new `amy use` verb.
Android Lint catches API-level, deprecation, accessibility, and resource
issues that spotless and unit tests miss. Run lint on the same variants
that get assembled (fdroidBenchmark, playBenchmark) before tests so a
lint failure surfaces quickly without consuming the full test budget.
Lint reports are uploaded as artifacts for inspection.
Resolution order in account mode (when --data-dir is not set):
1. --name X if given.
2. ~/.amy/current marker (set by `amy use X`).
3. Sole subdirectory of ~/.amy/ other than shared/.
4. Error — disambiguate with --name or `amy use`.
Single-account users skip the flag entirely (`amy whoami` Just Works
once one account exists). Multi-account users either pin one with
`amy use bob` (writes ~/.amy/current) or pass --name on every call.
The pinned account can be overridden by --name on a single command,
and cleared with `amy use --clear`.
`amy use` (no arg) prints the current pin plus the list of available
accounts. `amy use NAME` validates the name, requires the account dir
to already exist (else `no_account` with a creation hint), and
atomically writes the marker.
The auto-pick errors are deliberately self-explaining:
- 0 accounts → "no account at ~/.amy; create one with `amy --name X init`"
- 2+ accounts unpinned → "multiple accounts (alice, bob); pick one
with --name or `amy use <name>`"
- stale current marker → "current pins 'ghost' but ~/.amy/ghost
doesn't exist; rewrite with `amy use <name>` or pass --name"
`use` is dispatched before DataDir.resolve so it works even when the
auto-pick would fail — that's the whole point of having the verb.
The `name` field also lands in `whoami` output now (null in
--data-dir legacy mode).
Default on-disk layout becomes:
~/.amy/
├── shared/
│ └── events-store/ (lazy: created on first event)
└── <account>/ (created by `amy --name X init`)
├── identity.json
├── state.json
├── aliases.json
└── marmot/
Per-account state moves under `~/.amy/<account>/`; the events-store is
shared across accounts under `~/.amy/shared/`. The shared store is
safe to share for now because amy doesn't currently persist any
decrypted inner events to it (NIP-17 DMs unwrap-and-display in
DmCommands; MLS inner events go to the per-group .log under
<account>/marmot/groups/, not the event store). When that changes,
a follow-up will introduce a per-account private-events-store and a
composite reader.
A new `--name X` global flag selects (or creates) `~/.amy/X/`.
`--data-dir P` is preserved as a self-contained escape hatch — the
test harness and ad-hoc throwaway dirs use it, events-store stays
inside P. Pass exactly one of `--name` or `--data-dir`; passing both
or neither is bad_args (exit 2). Names must match
[a-zA-Z0-9_-]{1,64}; `shared` is reserved.
`amy init --name alice` writes a self-entry into
`<account>/aliases.json` ({"alice":"npub1…"}) so future commands and
the planned `amy alias add` / dm-recipient resolver can refer to the
account by name. The init result map gains a `name` key (null in
legacy mode).
Open question for a follow-up: when only one account exists in
~/.amy/, should `amy whoami` work without --name? Today it doesn't —
strict mode. Also pending: README rewrite, plus a sweep through
commands that print `data_dir` to also surface `name` where useful.
Three of the bugs that the SQLite review surfaced also live in the
filesystem-backed event store. Bringing both stores back to parity:
- NIP-01 lexical-id tiebreaker (FsSlots, FsEventStore): when two
replaceables / addressables share `createdAt`, the lexically smaller
id wins. The previous `existing.createdAt >= incoming.createdAt`
check rejected equal-timestamp inserts unconditionally — which
blocks the legitimate winner whenever it arrived second.
- delete(Filter()) safe-by-default (FsEventStore): an empty filter
used to enumerate every event and delete each one, so a stray
`delete(Filter())` would wipe the entire on-disk store. Now both
the single-filter and list-of-filters overloads short-circuit when
every filter is empty, matching the SQLiteEventStore contract.
- NIP-09 author check on tombstone install (FsEventStore,
FsTombstones): SQLite's `reject_deleted_events` trigger checks
`event_tags.pubkey_hash = NEW.pubkey_owner_hash`, so a stranger's
kind-5 with an `e`/`a` tag pointing at someone else's event must
not block them from re-publishing. The FS store used to install
the tombstone unconditionally and then read it back without an
author check.
- id tombstones still install (so they can fire when the deletion
arrives before its target), but `idTombstoneOwnerPubKey` is now
compared against the candidate event's owner pubkey at insert
time. GiftWrap parity preserved via FsIndexer.ownerPubKey, which
returns the recipient like the SQLite `pubkey_owner_hash`.
- addr tombstones are now only installed when
`addr.pubKeyHex == deletion.pubKey`, since the address itself
carries the owner identity.
Tests:
- FsSlotsTest: replaces "equal timestamp replaceable is rejected"
(which pinned the buggy behaviour) with two tests covering the
lexical-id tiebreaker in both insertion orders.
- FsParityTest: new same-`createdAt` tiebreaker tests for both
replaceable and addressable kinds, asserting FS and SQLite agree.
- FsDeletionTest:
- inverts "deletion by non-author does not cascade but still
installs id tombstone" — the legitimate owner must be able to
re-insert after the stranger's deletion.
- new test that a stranger's `a`-tag deletion does not block
the legitimate addressable owner from publishing a new version.
- FsEventStoreTest: new `delete with empty filter is safe` test
matching the SQLite-side contract added in batch A.
https://claude.ai/code/session_01X163Nr31vGkvAXoJ3JgMov
Batch B test-coverage gaps from the review:
- testTransactionRollsBackOnTriggerAbort: a multi-insert transaction
whose middle statement is rejected by `reject_deleted_events` rolls
back ALL inserts in the transaction, not just the failing one.
- testDeletionByThirdPartyDoesNothing: NIP-09 author check —
another user's deletion event must not remove the original.
- testKind5CanBeDeletedByAnotherKind5OfSameAuthor: pins the current
behavior that kind-5 events are not specially protected; a same-author
follow-up deletion can remove a previous one, and re-insertion of the
removed deletion is then blocked.
- testGiftWrapDeletionRequiresRecipient: NIP-59 GiftWraps key on the
recipient (p-tag), not the (encrypted) inner author. Sender and
unrelated third parties cannot delete; the recipient can.
- testVanishForDifferentRelayIsNoOp: a kind-62 vanish naming a
different relay must be stored but must not delete events on this
relay nor block new inserts (RightToVanishModule.shouldVanishFrom
contract).
- testFtsCleanedUpAfterReplaceableRotation: FTS rows are cleaned via
the AFTER DELETE trigger when an addressable is superseded — old
content must no longer match search.
- testVacuumAndAnalyseSmoke: VACUUM and ANALYZE run on a populated DB
without throwing and preserve existing rows.
- SqlSelectionBuilderTest: pins NotEquals(null) → IS NOT NULL,
empty IN → "1 = 0", equalsOrIn singleton/multiple paths.
https://claude.ai/code/session_01X163Nr31vGkvAXoJ3JgMov
`profile show` puts the parsed kind:0 content under `metadata` as a
Jackson `JsonNode` (`ProfileCommands.kt:114`). The text renderer only
recursed into `Map`/`List`, so the JsonNode fell through to
`toString()` and printed as a single quoted-JSON line:
metadata: {"name":"Alice","picture":"…",…}
Convert any embedded JsonNode to plain Java types via
`mapper.convertValue` once at the top of `renderText`, so the same
generic walk yields:
metadata:
name: Alice
picture: …
…
The walk handles nested cases (a hand-built Map containing a JsonNode
subtree, an ArrayNode inside a List, etc.). The `--json` shape is
untouched — Jackson's `writeValueAsString` already serialises JsonNode
natively.
- isExpired (#10): NIP-40 says an event is expired *once* `expiration`
is reached, and the SQL trigger uses `<= unixepoch()`. The Kotlin
pre-check used `<` (strict) so an event with `expiration == now`
passed the Kotlin check then failed in the trigger. Both layers now
use `<=`. Also applies to `isExpirationBefore` for consistency.
- transaction extension (#7): if the body throws *and* ROLLBACK also
throws, we now attach the rollback failure as a suppressed exception
instead of letting it mask the original cause. COMMIT is moved outside
the catch so a commit failure doesn't trigger a second ROLLBACK on
already-finalized transaction state.
- SeedModule.hasher (#8): the cache field is now a `kotlin.concurrent.
atomics.AtomicReference` (matches the pattern used in BleChunkAssembler
and BasicRelayClient) so the hasher publication is visible across
threads. The race itself is benign — the seed is stable, so two
concurrent computations produce identical hashers — but the prior
plain `var` had no visibility guarantee.
- delete(Filter()) (#12): documents the intentional asymmetry — `query`
on an empty filter returns everything, but `delete` on an empty
filter is a no-op (safe-by-default). New test pins the contract.
Tests:
- testInsertingEventExpiringExactlyNow: events with `expiration == now`
are rejected by both Kotlin and the trigger.
- testTransactionRollsBackOnException: a user transaction whose body
throws leaves the DB unchanged and still accepts new writes.
- testDeleteWithEmptyFilterIsSafe: empty-filter delete is a no-op.
https://claude.ai/code/session_01X163Nr31vGkvAXoJ3JgMov
Defaults to ANSI colour when stdout is a TTY (off when piped, off under
NO_COLOR=…, force-on under CLICOLOR_FORCE=1). Layout improvements that
apply to every command without per-command code:
- Sibling keys at each indentation level pad to the widest, so colons
line up YAML-style.
- Keys render bold; list dashes and "(none)"/"(empty)" markers render
dim.
- Booleans render as `yes` / `no` (green / red).
- Integer values under `*_at` keys render as
`2026-04-25 12:30:45Z (2m ago)` instead of a raw epoch second.
- Integer values under `*_bytes` keys (and `size`) render as
`7.0 KiB` / `1.2 MiB` / etc.
- Errors render `error: <code>: <detail>` with the prefix bold-red and
the code yellow.
The `--json` shape and exit codes are untouched; the smart formatting
only changes how scalars surface in the human-text mode.
Jackson's pretty-printer can emit \r\n on Windows depending on the
configured DefaultIndenter, so the previous one-sided normalization
still mismatched. Normalize both sides of the assert to LF.
Triple-quoted Kotlin strings inherit the source file's line endings,
so on Windows checkouts (CRLF) the expected JSON contains \r\n while
Jackson's pretty-printer always outputs \n. Normalize the expected
side with replace("\r\n", "\n") so the assertion is platform-neutral.
Surfaced by adding :quartz:jvmTest to the desktop CI matrix; previously
the bare 'test' lifecycle didn't resolve jvmTest in KMP modules so the
mismatch was never observed on Windows.
- NIP-09 (#2): a-tag and replaceable deletes now respect
`created_at <= deletion.created_at` so a stale deletion request
cannot remove a newer addressable that legitimately replaced it.
`+created_at` hint on the addressable path keeps the d_tag-selective
index in use.
- NIP-01 (#1): replaceable / addressable triggers now apply the
lexical-id tiebreaker — when two events share `created_at`, the
one with the lexicographically smaller id wins, matching the spec.
- Schema (#5): FullTextSearchModule.versionFinder probes FTS support
with a dummy table, but used to leave it behind. The first v1->v2
upgrade then failed because re-running create() would hit
"already exists". Now we drop the probe table immediately and
defensively clean up any stragglers.
- Schema (#6): onCreate / onUpgrade and the matching `setUserVersion`
are now wrapped in a single transaction so a partial migration
cannot leave the DB with mismatched user_version and schema.
- SQL DSL (#4): Condition.NotEquals(null) now produces `IS NOT NULL`
instead of `IS NULL`.
- Doc fix (#13): swapped vacuum/analyse comments now describe the
right command.
Tests: same-`created_at` tiebreaker for replaceables and addressables,
NIP-09 created_at window for a-tag deletes, schema drop+recreate
idempotency (covers the FTS dummy-table regression).
https://claude.ai/code/session_01X163Nr31vGkvAXoJ3JgMov
amy's default stdout is now a YAML-ish render of the underlying result
map; the previous single-line JSON contract moves behind a global
`--json` flag. Errors mirror the same rule (`error: <code>: <detail>`
on stderr by default, JSON `{"error":...,"detail":...}` under --json).
Exit codes (0/1/2/124) and the --json shape itself are unchanged —
only the default presentation flips.
- Replaces Json.writeLine / Json.error with mode-aware
Output.emit / Output.error. The same Jackson mapper is reused for
on-disk JSON via Output.mapper.
- Adds `--json` to the global flag set in Main.kt; honoured even when
argument parsing fails so error JSON keeps shape under --json.
- Updates the test harness wrappers (amy_a / amy_b / amy_d in
cli/tests/{headless,dm,cache}/) and the few direct $AMY_BIN call
sites whose stdout is consumed via $() / 2>&1 — they now pass
--json so the existing jq pipelines keep working.
- Rewrites the README "Output contract" and DEVELOPMENT design
principles to describe the new default, and clarifies that only
--json is the public API; the text shape is allowed to drift.
The bare 'test' task is ambiguous in KMP modules (candidates include
testAndroid and testAndroidHostTest). Switch quartz/commons/nestsClient
to :jvmTest so the desktop matrix unambiguously runs the JVM test
target on each OS. cli and desktopApp remain on :test (pure JVM).
nestsClient is a KMP module with both JVM and Android targets. Add it
to the desktop matrix's explicit test list so its JVM tests run on all
three desktop OSes alongside quartz/commons/cli/desktopApp.
The merged test-and-build matrix hid Android behind an OS-keyed matrix
leg, so the workflow graph no longer showed a distinct Android node.
Restore visibility by splitting into two parallel jobs after lint:
- build-desktop (matrix: ubuntu/macos/windows): runs JVM tests for the
KMP/JVM modules (quartz, commons, cli, desktopApp) and packages the
native installer per OS.
- test-and-build-android (ubuntu): runs the full test sweep and
assembleBenchmark, uploads APKs and reports.
Trade-off: Ubuntu spins up two parallel runners instead of one, but
wall time is unchanged and Android is once again a visible node.
The profile picture thumbnail cache pre-resized every source to a square
256x256 via createScaledBitmap, stretching non-square avatars. The cache
hit path returned the pre-squashed bitmap with isSampled=true, so the
ContentScale.Crop at the composable could not undo it.
Fuses center-crop + scale into a single Bitmap.createBitmap call with a
scale matrix so the cached thumbnail matches the CircleShape +
ContentScale.Crop render contract with one allocation. try/finally
guarantees the source bitmap is recycled even if createBitmap throws.
Bumps the cache subdir to profile_thumbnails_v2 so existing squashed
thumbnails are abandoned and regenerated, and reclaims the orphaned v1
directory on first init.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The GroupEventResult sealed class gained a ProposalStaged variant for
standalone Proposal events that get parked in the pending pool without
advancing the epoch. The when in DecryptAndIndexProcessor.add wasn't
exhaustive, breaking compileFdroidDebugKotlin and compilePlayDebugKotlin.
Logs the result at DEBUG since there's no UI work to do until a later
Commit references the proposal by hash.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The PROPOSAL branch of `MlsGroup.decrypt` previously threw
`IllegalStateException("Standalone PrivateMessage proposals not yet
supported")`. That worked in the openmls-as-Whitenoise topology because
MDK emits SelfRemove as a PublicMessage — but any peer using the
AlwaysCiphertext wire-format policy (RFC 9420 §6.3.2) wraps standalone
proposals in PrivateMessage, and we'd hard-fail on first contact.
Implements the PROPOSAL branch symmetrically with the existing COMMIT
branch:
1. Decode the Proposal struct from PrivateMessageContent (no length
prefix), read signature<V>, drain zero-padding.
2. Restrict to SelfRemove (mirrors `receivePublicMessageProposal`'s
policy — the only standalone proposal type that needs to ride
outside a commit; widening is one-line if interop demands it).
3. Verify the FramedContentTBS signature with `wire_format =
PRIVATE_MESSAGE` against the sender's leaf signature_key.
4. Stage the proposal in `pendingProposals` together with the encoded
AuthenticatedContent (wire_format ‖ FramedContent ‖ signature) so
a subsequent inbound commit folding it in by ProposalRef can
resolve via the §5.2 hash. PrivateMessage AC bytes carry no
membership_tag — auth = signature only.
Adds a symmetric `encryptProposalAsPrivateMessage` helper (internal,
mirrors `encrypt` for application data) so tests can exercise the
inbound path with a real wire frame produced by the same code path
peers use.
2 new tests: round-trip Bob→Alice SelfRemove via Welcome flow with
independent MlsGroup instances, verifies the proposal lands in
Alice's pending pool with AC bytes captured; and rejection of a
non-SelfRemove standalone PrivateMessage proposal (PSK in the test).
Marmot test suite green; marmot-interop-headless 16/16.
Closes audit gap #1.
https://claude.ai/code/session_013VYkpz8P1mPh9Ejxy9anhJ
idx/tag/<name>/ used to bucket every tag value under a 16-hex Murmur
hash, so `ls idx/tag/p/` showed opaque hex even though pubkey values
are perfectly safe filenames. Now the directory name is:
- the raw tag value, when it's filesystem-safe across Linux/macOS/
Windows: 1..180 bytes, printable ASCII (0x21..0x7e), none of
`/ \ : * ? " < > |`, no leading dot or `_h_`, no trailing dot/space;
- otherwise `_h_<hashHex(murmur)>`. The `_h_` sentinel can never
collide with a raw value (raw values are forbidden from starting
with it), so both forms safely live in the same parent dir.
Common cases keep their raw form and become directly inspectable:
ls idx/tag/p/<your_pubkey>/ — every event that p-tagged you
ls idx/tag/e/<event_id>/ — replies / reactions to an event
ls idx/tag/t/nostr/ — every kind-1 with #nostr
ls idx/tag/k/30023/ — k-tag pointers to articles
ls idx/tag/g/drt3n/ — geohash mentions
Routes through the _h_ bucket: emojis, URLs (slashes), `a`-tags
(colons), free-form `alt` text, anything ≥ 180 bytes, anything that
breaks Windows reserved-name rules. The hash is still seed-salted
Murmur64 (parity with the previous bucket), so collisions are caught
by FilterMatcher post-filter just like before.
Writer (FsIndexer.pathsFor) and reader (FsQueryPlanner.firstTagKey)
both route through FsLayout.tagValueDirName, so they always agree.
Tests: 5 new in FsQueryTest covering raw ASCII tags landing under
named dirs, p-tag pubkeys keeping their raw form, emoji and URL tags
falling back to _h_, and round-trip queries hitting both buckets
correctly. 122 fs tests green.
No migration: existing stores must `amy store scrub` once after the
upgrade — old purely-hashed tag dirs become unreachable, and scrub
rebuilds idx/ from canonicals using the new naming.
Two DoS surfaces in the inbound path:
**#8 — unbounded forward-ratchet on PrivateMessage decrypt.** A
malicious sender (or any peer who can put bytes in the wire frame)
controls the `generation` field of an inbound PrivateMessage. The
SecretTree was happy to fast-forward the sender's application or
handshake ratchet by however many steps that field implied — every
step costing one HKDF-Expand. A single packet with `generation =
2^31` would have pinned a CPU at SHA-256 for minutes. The skipped-key
cache (`MAX_SKIPPED_KEYS = 1000`) bounds memory but NOT compute: it
just stops *caching* past the cap, the ratchet keeps walking.
Adds `MAX_RATCHET_STEPS_PER_CALL = 4096` and rejects any decrypt that
asks for a larger jump from the sender's current head, applied at
both `applicationKeyNonceForGeneration` and
`handshakeKeyNonceForGeneration`. 4096 leaves room for legitimate
catch-up (mobile waking from a long sleep) while bounding the
worst-case per-packet cost to ~4096 SHA-256 invocations.
**#11 — oversized PrivateMessage AAD allocation.** The underlying
[TlsReader] already caps every opaque<V> read at 1 MiB, so the
ciphertext field is bounded — but `authenticated_data` and
`encrypted_sender_data` were also allowed up to 1 MiB even though
both fields legitimately carry a few hundred bytes at most. A
pre-verification frame would force ~3 MiB of allocation per inbound
packet. Tightens both fields to 64 KiB at PrivateMessage decode
(below TlsReader's global cap) so the per-frame floor is predictable.
2 new tests: `secretTree_rejectsRatchetJumpsBeyondCap` exercises the
boundary (4096 OK, larger throws with a "jump too large" message)
and `privateMessage_rejectsOversizedAuthenticatedData` hand-builds
a frame with a 65 KiB AAD field and confirms rejection. Marmot
test suite green; interop 16/16.
Closes audit gaps #8 and #11.
https://claude.ai/code/session_013VYkpz8P1mPh9Ejxy9anhJ
Two RFC 9420 hardening gaps in the inbound path:
**#3 — UpdatePath silent decrypt failures.** When `processCommit` was
handed a commit whose UpdatePath either (a) had no node at our common
ancestor with the sender or (b) had no ciphertext encrypted to a node
in our copath resolution, the path-decrypt block silently fell through
and left `commit_secret = 0`. The downstream confirmation_tag check
caught it and rolled back, but the rollback surfaced as a generic
"ConfirmationTagMismatch" instead of pointing at the real cause —
a tree-shape mismatch between our view and the sender's. Now hard-fail
with a precise error naming the indices involved.
**#6 — parent_hash chain verification on Welcome.** `processWelcome`
was checking the GroupInfo signature and the GroupContext.tree_hash,
but neither gates a forged parent_hash inside a COMMIT-source leaf —
the tree_hash check only proves the ratchet_tree extension matches
the bytes the signer signed, not that the stored parent_hash values
are consistent with the tree shape. A peer that DOES validate
parent_hash (per RFC 9420 §7.9) would reject every commit produced
from such a tree, splitting the group on the next epoch.
Adds `verifyTreeParentHashesForJoin(tree)` as a static-tree
counterpart to `verifyParentHash` (which only handles the
post-UpdatePath dynamic case). For each COMMIT-source leaf, recompute
the parent_hash chain top-down on the leaf's filtered direct path and
compare to the stored value. Returns null on success or a human-
readable mismatch reason. Wired into `processWelcome` right after the
tree_hash check, before any capability or key-schedule work.
Also moves `encodeParentHashInput` into the companion object so the
static and instance variants share one TLS encoder, and adds an
`exportTreeBytes()` test accessor.
2 new tests: trivial single-member tree accepts; a tampered
COMMIT-source parent_hash on a 3-member tree is rejected with a
message naming the offending leaf. Quartz marmot tests green;
marmot-interop-headless 16/16. (Two unrelated NostrClient network
tests fail under the full :quartz:jvmTest run — pre-existing flake,
no MLS code touched.)
Closes audit gaps #3 and #6.
https://claude.ai/code/session_013VYkpz8P1mPh9Ejxy9anhJ
When a group installs a `required_capabilities` extension (every Marmot
group does, listing MarmotGroupData=0xF2EE, SelfRemove=0x000A, Basic
credential), every member's leaf MUST advertise those types in its own
[Capabilities]. We were validating none of that:
- `applyProposalAdd` checked version + ciphersuite but never matched the
new KP's capabilities against the group's required_capabilities. A
non-conformant member silently joined; the next commit that touched
their leaf got rejected by spec-conformant peers, splitting the group.
- `processWelcome` similarly never checked the joiner's own KP against
the group's required set, nor did it sweep existing members' leaves.
A misconfigured GroupInfo signer could invite us into an incoherent
group whose first commit we'd silently reject forever.
Adds two helpers in `MlsGroup.Companion`:
- `findRequiredCapabilities(extensions)`: decodes the §7.2 struct from
a GroupContext extension list, or returns null if absent.
- `requireCapabilitiesMeetRequirements(caps, req, who)`: throws with a
specific (extensions=, proposals=, credentials=) diff naming the
missing types — turns silent interop breaks into one debuggable line.
Wires the gate in two places:
- `applyProposalAdd` rejects the Add proposal if the new leaf falls
short of the group's required set.
- `processWelcome` rejects the join if either (a) our own KP doesn't
meet the group's requirements or (b) any existing member's leaf
doesn't — the latter catches a malformed GroupInfo at join time.
5 new tests: round-trip decoding, rejection on missing extension /
proposal, acceptance when caps are a superset, and an end-to-end
`addMember` rejection of a hand-crafted KP with SelfRemove stripped
(re-signed so the rejection is from the capability gate, not the
signature check). Quartz test suite green; marmot interop 16/16.
Closes audit gaps #4, #5, #10.
https://claude.ai/code/session_013VYkpz8P1mPh9Ejxy9anhJ
assert_eq is (actual, expected, test_id, note); first version of the
script had test_id and actual swapped, so even passing assertions
showed up as fails with confusing "got T2.source" messages. Now all
21 assertions pass with the right arg order. Each successful assertion
also fires record_result pass so the results table covers them
(previously the tests passed but were invisible — only the explicit
record_result calls showed up).
Verified end-to-end on a real local nostr-rs-relay:
21 passed, 0 failed, 0 skipped (of 21)
Coverage:
T1.* — store stat reports kind histogram + disk usage after publish
T2.* — self profile show is served from cache by default
T3.* — --refresh forces source: relays
T4a/b — B's first lookup of A is a relay miss; second is a cache hit
T5.* — relay list reads URLs back from local kind:10002/10050/10051
T6 — relays.json is gone from both data-dirs
T7.* — store maintenance verbs work without an identity
Three changes that go together:
1. Reconcile cli/plans/2026-04-24-file-event-store-{overview,nips}.md
with shipped reality: code lives in quartz/jvmMain/, not commons/;
data dir is <data-dir>/events-store/, not <root>/events/.
2. New StoreCommands wired as `amy store …`:
- stat → events count, kind histogram, disk bytes,
oldest/newest createdAt. Pure read, no Context
(skips identity check).
- sweep-expired → wraps store.deleteExpiredEvents(); reports
{swept, remaining}.
- scrub → wraps store.scrub() to rebuild idx/ from
canonicals.
- compact → wraps store.compact() to drop dangling idx/.
All four open the FsEventStore directly (no Context, no identity
needed) — they're store-only operations.
3. New e2e harness at cli/tests/cache/cache-headless.sh that boots a
local nostr-rs-relay, two amy identities (A + B), and asserts:
T1 — store stat reports non-empty store after publish-lists +
profile edit, with kind:0 and kind:10002 present.
T2 — A's `profile show` is `source: "cache"` by default.
T3 — `--refresh` forces `source: "relays"`.
T4 — B's first `profile show <A_NPUB>` is a relay miss; second is
a cache hit (proves drain populates the local store and
subsequent reads serve from disk).
T5 — `relay list` reads URLs back from the local kind:10002 /
10050 / 10051 events.
T6 — relays.json no longer exists in either data-dir.
T7 — store stat / sweep-expired / scrub / compact all run
without an identity present.
Same pattern as cli/tests/dm/dm-interop-headless.sh — reuses the
nostr-rs-relay infrastructure from cli/tests/marmot/setup.sh.
Vitor Pamplona