When the local user leaves a Marmot MLS group, MarmotManager already
wipes MLS state, the relay subscription and the persisted message log,
but the chatroom (and its decrypted inner messages) lingered in
marmotGroupList until the UI happened to call removeGroup. The CLI
leave path skipped that step entirely, and removeGroup itself only
dropped the chatroom from rooms — it left noteToGroupIndex entries and
the chatroom's own messages set in place, so notes stayed strongly
referenced and kept appearing in the Notification feed.
Move the in-memory cleanup into Account.leaveMarmotGroup, and have
MarmotGroupList.removeGroup also clear the chatroom's messages and the
note→group index. LocalCache holds notes weakly, so cutting the strong
refs is enough — GC reclaims them and the existing acceptableEvent
filter (which keys off marmotGroupList.rooms) hides anything still
in-flight.
`MarmotInboundProcessor.processGroupEvent` added every event id to its
dedup set regardless of result, including `UndecryptableOuterLayer`. The
handler buffers those events for a post-commit retry, but the replay hit
the Duplicate early-return and skipped MLS decryption — so future-epoch
messages and group-metadata commits that arrived before their epoch's
commit were silently dropped until an app restart re-fetched them.
This is interop test 12 (offline catch-up): messages 6-8 and the rename
commit only appeared after restarting Amethyst.
Skip the dedup write on `UndecryptableOuterLayer` so retries actually
re-attempt decrypt once the epoch advances. Memory is still bounded by
the handler's per-group pending buffer (64 events, FIFO-evicted).
The notification filter accepted any note whose `inGatherers` contained
a MarmotGroupChatroom, regardless of which account's list that chatroom
belonged to. Since notes live in the global LocalCache and accumulate
gatherer references across account switches (and are not cleaned up on
leave), old messages stayed in Notifications after switching accounts
or leaving a group.
Mirror the NIP-17 DM guarantee: only accept a Marmot message when the
gatherer is the same instance currently held by the active account's
`marmotGroupList.rooms[nostrGroupId]`.
Replaces the two-row, mixed-control filter UI under the search bar with a
single segmented row for scope (All/People/Notes) and a Tune icon button
that opens a ModalBottomSheet containing Source, Follows-only, and Sort
controls. A primary-color dot on the icon indicates non-default filter
state.
Also renames SearchSortOrder.DEFAULT_EVENT/DEFAULT_PEOPLE to
EVENT_DEFAULT/PEOPLE_DEFAULT for consistency with EVENT_OPTIONS/
PEOPLE_OPTIONS, and routes the sheet's reset button through these
canonical constants so the UI default cannot drift from the ViewModel.
Android search previously dumped users, notes, hashtags, relays, and 3 channel
types into a single untoggled list and silently relied on NIP-50 relay search.
This adds a second-row control strip so users can pick what they're looking for
and how results are sourced/ranked.
Commons:
- New SearchScope { ALL, PEOPLE, NOTES } and SearchSource { LOCAL, RELAYS }
- SearchSortOrder: add POPULAR; EVENT_OPTIONS now [RELEVANCE, NEWEST, POPULAR]
Android SearchBarViewModel:
- scope / source / followsOnly / sortOrder StateFlows
- searchResultsUsers + searchResultsNotes respect scope, follows, and sort
- POPULAR sorts notes by Note.zapsAmount (descending)
- Follows-only filters authors (includes replies since filter is by author)
- directEventResolver: bech32 nevent/note/naddr -> Route.Note (auto-navigate)
- updateDataSource skips relay emit when source == LOCAL
Android SearchScreen:
- Second row: [All | People | Notes] SegmentedButton + Sort dropdown
- Third row: [Local | Relays] SegmentedButton + Follows-only chip
- bech32 hit triggers nav.nav(route) and clears the search
Desktop parity deferred - Desktop already ships an advanced panel that covers
most of this (kinds/authors/dates/hashtags/language/exclude terms + sort +
bech32 direct lookup). The new enums live in commons so Desktop can adopt.
Build note: gradle 9.3.1 can't be downloaded in this environment, so Android
and Desktop compilation were not verified here.
R8 in playBenchmark (minifyEnabled) stripped the no-arg <init> from
WorkDatabase_Impl because -keepnames preserves names but not members.
Room instantiates generated *_Impl subclasses reflectively via
Class.getDeclaredConstructor(), so startup crashed with
NoSuchMethodException in androidx.startup.InitializationProvider.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ExoPlayer's URI-based content-type inference only fires for http(s)
schemes + known extensions, so HLS playback fails in two cases Amethyst
ships regularly:
1. NIP-71 video events with BUD-10 blossom URIs. Hash-based URIs carry
no extension and the imeta's application/vnd.apple.mpegurl never
reaches the MediaItem — MediaItemCache dropped it.
2. Bare BUD-10 URIs pasted into kind 1 posts, which have no imeta at
all. Even though the URI contains `.m3u8` in its path, ExoPlayer's
inference skips custom schemes (`blossom:`).
Forward a normalized mimeType onto MediaItem.Builder.setMimeType so
DefaultMediaSourceFactory routes to HlsMediaSource regardless of scheme
or extension:
- Normalize IANA Apple HLS variants (application/vnd.apple.mpegurl,
application/x-mpegurl, audio/x-mpegurl, audio/mpegurl) to
MimeTypes.APPLICATION_M3U8.
- Fall back to `.m3u8` URI detection when no imeta mimeType is
present. Matches the existing isLiveStreaming(url) heuristic.
Adds diagnostic logs at MediaItemCache.compute and
CustomMediaSourceFactory.createMediaSource so the routing decision is
observable in logcat, matching the diagnostic-logs pattern already used
elsewhere in playback.
- Persist relay list events (kinds 10050/10007/10006) as JSON to
java.util.prefs.Preferences with per-account key isolation
- Load persisted relay configs on startup before bootstrap subscription
- Validate loaded events (kind + pubkey check), 8KB guard on writes
- Fix SearchScreen relay count: "0 of 1" not "0 of 7" — uses searchRelays
- Fix FeedScreen relay count: shows feed relay count, not all connected
- Per-screen relay picker dialogs: Dns icon on Feed and Search screens
opens AlertDialog wrapping existing editors (Nip65RelayEditor,
SearchRelayEditor) — no new composable files
- Fix created_at dedup: use >= for replaceable event semantics
- Fix setters: use TimeUtils.now() not Long.MAX_VALUE
- Add consumePublishedEvent() for local immediate update after publish
- Remove stale "not loaded" warnings from Search/Blocked editors
- Fix FeedHeader type: Set<NormalizedRelayUrl> not Set<Any>
- Fix picker LaunchedEffect(Unit) to not overwrite user edits
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Test 12 (offline catch-up) was almost passing: the retained-epoch
fallback (c88923a3) brought through the 5 epoch-1 application messages
the user sent before adding C, but 3 epoch-2 messages + the rename
commit still went missing. Only on app restart did they appear — the
relay re-delivered and the replayed arrival order happened to put the
epoch-2-advancing commit before the epoch-2 events.
Root cause: kind:445 events arrive from the relay in subscription
order, not epoch order. When B drives an offline-catch-up sequence
like
msg-1..5 (epoch 1) → add C (epoch 1→2) → msg-6..8 (epoch 2) →
rename (epoch 2→3)
the receiver can easily see {msg-6, msg-7, msg-8, rename, add-C,
msg-1..5}. Everything from the rename onward arrives before the
add-C commit, so the outer ChaCha20-Poly1305 layer fails with
UndecryptableOuterLayer — we don't have the epoch-2 exporter yet.
Quartz correctly reports the failure, but the receiver just logged
"likely from before our join" and dropped the event on the floor.
Nothing retried when the add-C commit eventually landed and the
epoch caught up.
Fix it at the GroupEventHandler level, where we can both see the
original GroupEvent and hook into every CommitProcessed:
- Keep a bounded per-group ArrayDeque<GroupEvent> of events that
failed with UndecryptableOuterLayer. FIFO-evict at 64 entries so
a genuinely pre-join stream can't pin unbounded memory.
- On every CommitProcessed for a group, drain that group's queue
and re-run each event through the normal add() path. A successful
retry may itself emit another CommitProcessed and trigger recursive
draining — that's fine, epochs are strictly monotonic so recursion
depth is bounded by the queue size.
- Concurrent-safe: all queue mutation is guarded by a Mutex.
Ambient discriminator: we can't tell "epoch too old, from before our
join" from "epoch too new, commit hasn't arrived yet" just from
UndecryptableOuterLayer — both report it. Buffering both is fine:
truly-pre-join events stay buffered until FIFO eviction pushes them
out (no correctness problem, just a few dozen bytes of memory for a
bounded time).
MIP-03 inner events inside a kind:445 GroupEvent are unsigned (sig is
empty or placeholder), so LocalCache's `justVerify()` fails and
`consume(ReactionEvent)` skips the branch that attaches the reaction to
its target note via `addReaction()`. The same applies to kind:5
deletions. Side-channel inner events from other Marmot clients
(WhiteNoise in particular) were therefore silently dropped by
LocalCache, so an inbound emoji reaction never appeared on the target
chat bubble.
NIP-17 has the same "unsigned inner rumor" shape and already hands the
inner event to `justConsume(..., wasVerified = true)`. Align Marmot with
that contract at the two inbound call sites:
- `GroupEventHandler.add` (fresh kind:445 from relay)
- `Account` restore-from-disk loop (persisted plaintext at app start)
Authenticity is already guaranteed by the MLS layer:
`MarmotInboundProcessor.processPrivateMessage` rejects any inner event
whose `pubkey` doesn't match the MLS sender credential identity before
ever emitting an `ApplicationMessage`.
https://claude.ai/code/session_01KhVtLJAEkNCwpRWUUTqDFM
The `tryDecryptWithRetainedEpoch` fallback in `MlsGroupManager.decrypt`
exists precisely to cover the interop harness's Test 12 scenario —
application messages encrypted under epoch N arriving at a receiver
that has already processed a commit advancing to N+1. The primary
path throws "Message epoch X doesn't match current epoch Y", then the
manager iterates the retained epoch window and tries each.
Two bugs kept the fallback from ever succeeding:
1. sender-data ciphertext sample length. The primary decrypt path uses
`MlsCryptoProvider.HASH_OUTPUT_LENGTH` (32 bytes, = KDF.Nh for
HKDF-SHA256, per RFC 9420 §6.3.2); the retained path was using
`AEAD_KEY_LENGTH` (16). The comment on the primary path — "Using
AEAD.Nk here made sender-data decryption fail against every
spec-compliant sender" — was the same fix just never propagated
to this branch. sender-data AEAD always failed, the try/catch
swallowed it, and the fallback returned null.
2. content plaintext was returned raw. AEAD output is a
PrivateMessageContent struct (§6.3.1):
`applicationData<V> || signature<V> || padding`.
The primary decrypt path parses this with `pmcReader.readOpaqueVarInt()`
to extract applicationData; the retained path returned the whole
struct. Callers saw a length-prefixed blob with signature + zero
padding glued on — obvious in a raw diff, invisible in a pass/fail
test that only checked "decrypted is not null".
Fix both in `tryDecryptWithRetainedEpoch`. Existing quartz↔quartz
tests still pass because they don't exercise out-of-order decrypt —
this branch was effectively dead code before.
Add testDecryptRetainedEpoch_ApplicationMessageAfterCommit as a
regression: Bob encrypts at epoch N, Alice advances to N+1 via
add-member, Bob's epoch-N message then arrives at Alice — must round-
trip through the retained-epoch fallback and match the original
plaintext exactly.
Adds a star toggle next to each member in MarmotGroupInfoScreen that
admins can use to promote a non-admin to admin or demote a fellow
admin. Both paths show a confirmation dialog and publish a
GroupContextExtensions commit that rewrites admin_pubkeys. The toggle
is hidden for non-admin viewers and for the signer's own row (self
demotion must go through the leave flow). The demote button is also
hidden when the member is the only remaining admin so we never hit the
MIP-03 admin-depletion guard in MlsGroup at commit time.
Account.grantMarmotGroupAdmin / revokeMarmotGroupAdmin reuse
updateMarmotGroupMetadata so the new admin list ships alongside the
signer's outbox relays, keeping the canonical relay set consistent
with other metadata commits.
Test 13 (KeyPackage rotation) fails with "no new KP event_id observed"
— the interop log shows three `needsKeyPackageRotation=true` triggers
(once per group join), each of which invokes
`account.publishMarmotKeyPackages()` from `processMarmotWelcomeFlow`,
but no subsequent `kind:30443` publish ever appears in the logcat.
`publishMarmotKeyPackages` was completely silent: no-op guards, the
needsRotation check, the rotated-events count, and the per-event
publish all ran without a log line. We can't tell whether the rotation
produces events that never reach a relay, produces zero events,
short-circuits on needsRotation=false, or is never called at all.
Mirror the addMarmotGroupMember log pattern: entry guards (manager
null / not writeable), the needsRotation + relay count at decision
time, the rotateConsumedKeyPackages produce-count, and a per-event
publish line with id + target relay list. Next rerun of Test 13 will
tell us which branch is silently dropping the rotation.
Interop Test 06 (wn removes a member from a group Amethyst is in) was
failing with:
GroupEventHandler.add: ERROR Failed to apply commit:
UpdatePath node count (1) doesn't match direct path length (2)
Repro scenario: 3-member group [B=leaf 0, C=leaf 1, A=leaf 2]; B (admin,
committer) removes C. After the Remove proposal applies, leaf 1 is
blank but leaf 2 (A) is still occupied — so leaf_count stays at 3 per
RFC 9420 §7.8 and the sender's direct path has length 2 ([node 1,
root]).
Per RFC 9420 §4.1.2 and §7.9 the **filtered direct path** drops any
parent node whose child on the copath has an empty resolution:
encrypting to such a parent is equivalent to encrypting to its only
non-blank child, which is already on the direct path. In our scenario
the parent at level 1 (parent of leaves 0, 1) has a blank leaf 1 on
the copath — empty resolution — so it's filtered out. Filtered
direct path length is 1, and that's what openmls / whitenoise-rs emit
in `UpdatePath.nodes<V>`.
Quartz was always using the unfiltered direct path on both sides
(send + receive + parent-hash chain + path-secret decrypt). That
worked for quartz↔quartz (both sides agreed), but broke the moment a
spec-correct sender (wn) sent a filtered UpdatePath into a quartz
receiver. Parent-hash chain is further affected because §7.9.2 walks
the filtered path — even if the size check passed, the hashes would
not match.
Fix it throughout:
- RatchetTree.filteredDirectPath: new helper returning parallel
(filteredDirectPath, filteredCopath) with the empty-resolution
entries dropped. Uses the existing `resolution()` helper, which
already handles unmerged_leaves per §4.1.2.
- RatchetTree.applyUpdatePath: size-check + target the FILTERED path.
- MlsGroup.commit (sender): emit one staged path node per filtered
level; encrypt path secrets using the filtered copath; patch parent
hashes into the filtered positions only.
- MlsGroup.processCommitInner (receiver): the UpdatePath node index
aligns to the filtered direct path, so the common-ancestor lookup
must find the filtered index to pick the right ciphertext + copath
resolution. Use the unfiltered index only for counting KDF steps
to the root (path_secret chain advances one step per unfiltered
level regardless of filtering — the chain is continuous; filtering
only decides which levels emit a UpdatePathNode, not how many KDF
steps separate them).
- MlsGroup.computeSenderParentHashes: walk the filtered direct path.
Map each filtered node back to its unfiltered level so the
preUpdateSiblingHashes lookup (indexed by unfiltered level) still
resolves. This makes the parent_hash chain agree with §7.9.2 and
with what openmls computes on the other side.
- MlsGroup.verifyParentHash: short-circuit on empty filtered path
rather than empty unfiltered path.
New regression test: testRemoveMiddleLeaf_ReceiverAcceptsCommit
reproduces the 3-member remove-middle scenario end-to-end within
quartz. It passed before this patch (because both sides were
unfiltered and agreed with each other), and it passes after this
patch (because both sides are now filtered and still agree) — the
compatibility win is that quartz now also agrees with
openmls/wn-produced commits of the same shape.
Also fix an unrelated script bug in marmot-interop.sh: the
discover_a_relays SQL probe was falsely reporting "wn has NO cached
relay entries for A" even when welcome delivery was working, because
the query assumed users.pubkey stored as BLOB but wn stores it as
TEXT (hex) in the current schema. Accept either encoding.
The first pass only fixed `group add`. Auditing the rest of the CLI
against MIP-00..03 turned up three more spots where `amy` either
queried the wrong relays or silently skipped a required advertisement:
1. `marmot key-package check <npub>` used `anyRelays()` — i.e. the
inviter's configured relays — so checking for a KeyPackage on a
user who advertises `kind:10051` somewhere we don't know about
always returned `not_found`. Now runs RecipientRelayFetcher against
bootstrap seeds and fetches from the union of (target kind:10051,
target kind:10002 write, bootstrap). Emits `found_on` so callers
can see which relay served the hit.
2. `await key-package <npub>` had the same bug inside the poll loop.
Resolved once up front, then the loop fetches from the target's
advertised relays every tick. Throws an `AwaitTimeout` early if no
relays can be discovered at all, instead of silently polling void.
3. `relay publish-lists` published kind:10002 + kind:10050 but never
kind:10051. Per MIP-00 the KeyPackage Relay List is how other
Marmot clients discover where our KPs live — without it the
`key_package` bucket on disk is invisible to anyone else. Now also
publishes kind:10051; falls back to the NIP-65 set if the bucket
is empty so we never advertise an empty list. (`amy create`
already publishes it via AccountBootstrapEvents.)
Docs: cli/README adds a "Relay routing" section that lists the exact
relay set used for publish vs fetch of every Marmot event kind, plus
the bootstrap-pool definition, so agents + interop-test authors can
reason about cross-user reachability without reading the code.
WhiteNoise threads its kind:445 payloads across three inner kinds:
kind:9 for chat, kind:7 for emoji reactions, kind:5 for unreacts. The
Amethyst ingest pipeline was routing every inner event into the group
chatroom feed, so a kind:7 reaction rendered as a chat bubble whose
only content was the emoji, quoting the liked message via the target
`e` tag. That looked identical to a threaded reply. The actual kind:9
reply from `wn messages send --reply-to` never arrived, which made the
two look swapped in the UI.
Three changes to untangle this:
- MarmotGroupList.addMessage/restoreMessage now skip inner events with
kind:5 and kind:7 before they enter `MarmotGroupChatroom.messages`.
The reaction is still consumed by LocalCache so it attaches to the
target note's reaction row, and the deletion still revokes that
reaction — they just don't appear as standalone bubbles.
- LocalCache.computeReplyTo learned to derive thread parents for
`ChatEvent` (kind:9) from plain NIP-10 `e` tags in addition to the
existing NIP-18 `q` tag path. WhiteNoise emits `e`-tagged replies;
without this the reply bubble had no quote context in the feed.
- tools/marmot-interop/marmot-interop.sh: `wn messages send` exposes
its `reply_to` field through clap v4, which renames snake_case to
kebab-case by default. The script was passing `--reply_to`, which
clap rejected; the `|| true` + redirected stderr hid the error and
no reply was ever published. Use `--reply-to`.
https://claude.ai/code/session_01K3g1uWLhByoEdBS77zdF32
Post-v0.2 wn wraps `--json groups show` output in {"result": {...}},
so `jq '{name, epoch}'` was reading the top level where those keys
don't exist — Test 10 always logged "B state: {name: null, epoch: null}"
and the operator couldn't tell whether the race produced the expected
converged state on wn's side.
Peel the .result wrapper before extracting the fields; same pattern
already used everywhere else in the script that parses wn JSON.
`amy marmot group add` previously sent the Welcome gift wrap to the
inviter's own kind:10050 (ctx.inboxRelays()) and fetched the invitee's
KeyPackage from only the inviter's configured relays. Two users with
disjoint relay configurations could never successfully marmot each
other — the welcome landed somewhere the invitee never polled.
Mirror the Android Account.addMarmotGroupMember flow in the CLI:
- New RecipientRelayFetcher in quartz/marmot one-shot-drains a target
user's kind:10050, kind:10051 and kind:10002 from a seed relay set.
Replaceable-event semantics: newest created_at per kind wins. Guards
against relays echoing events authored by someone else.
- Context.bootstrapRelays() unions the inviter's configured relays with
AmethystDefaults (DefaultNIP65RelaySet + DefaultDMRelayList) so we can
discover strangers even when nothing overlaps with our own config.
- GroupAddMemberCommand now per-invitee: looks up their relay lists;
passes kind:10051 + kind:10002 write + bootstrap to KeyPackageFetcher;
publishes the welcome to kind:10050 (fallback to NIP-65 read, then
DefaultDMRelayList, then our outbox as belt-and-braces).
Group commits/messages (kind:445) continue to use the group's MIP-01
relay set — those were already correct. Exposes welcome_targets and
key_package_relays in the JSON output so callers can verify routing.
Before, every Marmot group landed unconditionally under the Known tab of the
Messages screen while the New Requests tab ignored groups entirely, so an
invite from a stranger appeared next to real conversations. The dedicated
Marmot group list screen split by ownerSentMessage only, so unreplied groups
where an admin was followed still showed as New Requests.
Replace both splits with a shared MarmotGroupChatroom.isKnown(followingKeySet):
- user has replied -> Known
- no known members and no messages -> New Requests
- otherwise Known iff any admin is in the follow set
Wire the combined Messages feeds (ChatroomListKnownFeedFilter and
ChatroomListNewFeedFilter) through the helper, invalidate dmNew on group list
changes so empty groups flow into New Requests, and have
MarmotGroupListScreen observe kind3FollowList so newly followed admins move
groups between tabs immediately.
The reverse-rename leg issued `wn_b groups rename` on Interop-02, but
Amethyst created Interop-02 — A is the admin, B is a plain member.
MIP-03's `enforceAuthorizedProposalSet` rejects any GroupContextExtensions
commit from a non-admin, so wn's rename was being dropped client-side,
the metadata commit never reached any relay, and Amethyst's UI (correctly
waiting for a kind:445 on Interop-02) never saw "Interop-02-reverse".
Split Test 07 into two separately-recorded legs so each uses a group
where the renamer is actually the admin:
07a — A (admin of Interop-02) renames to "Interop-02-renamed" via
the Amethyst UI; wn-side poll observes the new name.
07b — B (admin of Interop-03) renames to "Interop-03-renamed" via
`wn_b groups rename`; operator confirms Amethyst picked it up.
Each leg individually records pass/fail/skip so a missing GROUP_03
(Test 03 skipped) doesn't take out the whole test.
Test 06 was asking the operator to remove C from Interop-05 via the
Amethyst UI — but Interop-05 was created by B (wn), so B is the sole
admin and A is a plain member. MIP-03's authorization gate
(`enforceAuthorizedProposalSet` in `MlsGroup.kt:1842`) rejects any
Remove proposal from a non-admin, so the UI correctly fires "not an
admin" and the test fails against its own author's assumption — the
test was misaligned with the admin model, not the code.
Flip the flow: B (admin) issues the removal via `wn groups
remove-members <gid> <C_NPUB>`, and A / C observe the effect. That
still exercises the same intended invariants:
- B + A remain in the group, commit + epoch advance propagate.
- A's Amethyst UI shows C gone (visible confirm).
- A sends "after removing C" → B decrypts (expected).
- C cannot decrypt the new message (forward secrecy at the new
epoch — C's retained keys only open the pre-remove epoch).
The "Amethyst admin issues remove" path is already separately
covered by Test 02 (A creates the group → A is admin → A removes)
and Test 08 (promote A, then A issues an admin-only action). Test
06's focus is forward secrecy after remove, not who may initiate —
so having B drive it is the spec-correct shape.
Move the add-member search to a fixed position at the bottom of the screen
(matching the New Short Note pattern) so it stays visible regardless of how
far the user has scrolled through a large member list. Suggestions now render
above the input, and each suggestion row shows an explicit PersonAdd icon
button so the add action has a clear visual affordance.
Adds an optional trailingContent slot to ShowUserSuggestionList / UserLine
so callers can attach per-row actions without forking the component.
https://claude.ai/code/session_01JbRycm4g3LrqatnCVWVdGh
Vitor Pamplona