fix(marmot,cli,interop): interop-compatible Marmot flows and harness correctness
Five protocol-level fixes and a batch of harness correctness fixes to get
the headless Marmot/Whitenoise interop harness from 1/13 to 5/13 passing
cleanly, with the remaining failures all rooted in wn's per-account
serial event-processor retry backoff (which drops undecryptable
pre-membership commits after several minutes) rather than amy behaviour.
quartz + commons
----------------
* MarmotGroupData: hold CURRENT_VERSION at 2. mdk-core (the Rust MLS
engine used by whitenoise-rs) strict-rejects v3 payloads with
`ExtensionFormatError("Trailing bytes in NostrGroupDataExtension")`
— our v3 welcomes and GCE commits never apply, so every cross-client
group flow dies at welcome processing. We still parse v3 happily on
the way in; we just don't emit it until mdk publishes the
forward-compat fix MIP-01 mandates.
* MarmotManager.updateGroupMetadata: MERGE extensions instead of
REPLACING. RFC 9420 §12.1.7 says GCE proposals blow away the old
extension list; callers that pass only [marmot_group_data] dropped
[required_capabilities], which peers then reject. Preserve every slot
except the one we're updating.
* MarmotManager.createGroup: new optional `initialMetadata` parameter
that bakes MarmotGroupData into epoch-0 GroupContext.extensions
directly. Without it, creators had to publish a pre-membership
"bootstrap" commit that no later joiner could decrypt — each such
peer then burned their retry budget on an undecryptable kind:445
before seeing the real state. Threaded through MlsGroup.create /
MlsGroupManager.createGroup.
* MarmotManager.mlsGroupIdHex: new translation helper so any code
juggling the MIP-01 nostr_group_id (what amy indexes on) and the
MLS GroupContext groupId (what mdk indexes on) can cross-reference
them without reaching into MlsGroupManager directly.
amethyst module
---------------
* Account.leaveMarmotGroup: self-demote before SelfRemove per MIP-01,
and promote a surviving member to admin first if the caller is the
sole admin (otherwise we'd throw "admin depletion"). Matches the
cli/GroupMembershipCommands.leave flow.
cli (amy)
---------
* Context.syncIncoming: don't advance `giftWrapSince` on empty polls
(so the first-ever sync doesn't bump the cursor past every
past-timestamped wrap we've ever been sent), subtract 2 days lookback
when filtering (NIP-59 randomWithTwoDays gift wraps can have any
createdAt in the last 48h), and only advance `groupSince` for groups
we actually received events for.
* Context.syncIncoming: after ingest, if any Welcome consumed a
KeyPackage, rotate and publish a fresh one immediately. MIP-00
requires this — a KP can only be welcomed once and leaving the
consumed one on relays just means later senders invite us with a
bundle we no longer have private keys for.
* Context.resolveGroupId: accept either nostr_group_id (amy's primary
key) or the MLS GroupContext groupId (what wn emits) on every verb
that takes a group id. Wired through GroupAdd/Remove/Leave/Metadata
/Read, Message send/list, and all the await* verbs so harness
scripts never have to juggle both forms for a single group.
* GroupCreateCommand: bake initial metadata into epoch 0 (see the
quartz change above). Dropped the now-redundant bootstrap commit
publish and tightened the JSON output to include `mls_group_id`.
* GroupMembershipCommands.leave: self-demote admin before SelfRemove;
promote an heir if we're the only admin, otherwise the GCE would
deplete admins and the leave aborts.
* MarmotIngest.ingestGiftWrap: unwrap the sealed-rumor layer. NIP-59
wrap is gift-wrap(kind:1059) → seal(kind:13) → rumor; the old code
only unwrapped once and then checked `inner.kind == 444`, which is
always false because inner is actually the seal. Unseal once more
before the Welcome check. This single fix is what unsticks every
amy-side Welcome ingestion.
wn harness patches + scripts
----------------------------
* whitenoise-defaults-env.patch: honour $WHITENOISE_DISCOVERY_RELAYS in
`Relay::defaults()` (release builds otherwise bake damus.io / primal
/ nos.lol into every new account's NIP-65 / Inbox / KeyPackage
lists, which breaks publishing and prevents the inbox subscription
plane from ever reaching an operational state in a sandbox).
* setup.sh: sleep 2s after amy's initial kind:30443 publish so
nostr-rs-relay has a chance to fsync before wn's first targeted
discovery query. Without it wn's `keys check` races the relay's
WAL flush and intermittently returns NotFound.
* lib.sh: peel wn's `{"result": …}` wrapper in `jq_group_id`,
`wait_for_invite`, `wait_for_message`, `wait_for_member`. Post-v0.2
wn `--json` output nests everything under `.result` (and
`groups invites[]` nests further under `.group.mls_group_id`) —
these helpers were still pattern-matching on the flat shape, so
they returned empty strings for a perfectly good response.
* tests-{create,manage,extras}.sh: track both group IDs per test
(amy's nostr + wn's MLS), pass each CLI the id it understands, and
bump the post-commit wait timeouts to 90–120s so wn's exponential
retry backoff has time to work through the pre-membership commits
it can't decrypt and get to the ones it can.
https://claude.ai/code/session_016kAxdp6ubB5CnF9URhCEzP
This commit is contained in:
+16
-3
@@ -27,6 +27,7 @@ import com.vitorpamplona.quartz.marmot.mip02Welcome.WelcomeEvent
|
||||
import com.vitorpamplona.quartz.marmot.mip03GroupMessages.GroupEvent
|
||||
import com.vitorpamplona.quartz.nip01Core.core.Event
|
||||
import com.vitorpamplona.quartz.nip01Core.core.HexKey
|
||||
import com.vitorpamplona.quartz.nip59Giftwrap.seals.SealedRumorEvent
|
||||
import com.vitorpamplona.quartz.nip59Giftwrap.wraps.GiftWrapEvent
|
||||
|
||||
/**
|
||||
@@ -95,11 +96,23 @@ suspend fun MarmotManager.ingest(event: Event): MarmotIngestResult =
|
||||
|
||||
private suspend fun MarmotManager.ingestGiftWrap(wrap: GiftWrapEvent): MarmotIngestResult =
|
||||
try {
|
||||
val inner = wrap.unwrapOrNull(signer) ?: return MarmotIngestResult.Ignored
|
||||
if (!MarmotInboundProcessor.isWelcomeEvent(inner) || inner !is WelcomeEvent) {
|
||||
// NIP-59 wraps contain TWO encryption layers:
|
||||
// kind:1059 gift wrap → kind:13 sealed rumor → the rumor itself.
|
||||
// `GiftWrapEvent.unwrapOrNull` only peels the outer layer; when the
|
||||
// result is a [SealedRumorEvent] we must unseal it to reach the
|
||||
// kind:444 Welcome rumor. The old code checked `isWelcomeEvent` on
|
||||
// the seal (kind:13) and always took the Ignored branch, which is
|
||||
// why every inbound Welcome was silently dropped by the CLI and by
|
||||
// any non-Amethyst consumer.
|
||||
val rumor =
|
||||
when (val inner = wrap.unwrapOrNull(signer) ?: return MarmotIngestResult.Ignored) {
|
||||
is SealedRumorEvent -> inner.unsealOrNull(signer) ?: return MarmotIngestResult.Ignored
|
||||
else -> inner
|
||||
}
|
||||
if (!MarmotInboundProcessor.isWelcomeEvent(rumor) || rumor !is WelcomeEvent) {
|
||||
return MarmotIngestResult.Ignored
|
||||
}
|
||||
when (val result = processWelcome(inner, inner.nostrGroupId())) {
|
||||
when (val result = processWelcome(rumor, rumor.nostrGroupId())) {
|
||||
is WelcomeResult.Joined -> {
|
||||
MarmotIngestResult.JoinedGroup(
|
||||
nostrGroupId = result.nostrGroupId,
|
||||
|
||||
+40
-4
@@ -262,11 +262,24 @@ class MarmotManager(
|
||||
|
||||
/**
|
||||
* Create a new MLS group.
|
||||
*
|
||||
* When [initialMetadata] is non-null it is baked into epoch 0's
|
||||
* GroupContext.extensions. Later joiners' welcomes therefore carry the
|
||||
* group name / admin list / relays from the get-go and no separate
|
||||
* "bootstrap commit" needs to be published. The bootstrap-commit path
|
||||
* works in theory, but it produces a kind:445 encrypted with epoch 0's
|
||||
* exporter secret that no post-membership peer (amethyst or wn) has,
|
||||
* so each such peer wastes their commit-retry budget on an
|
||||
* undecryptable event before processing the real state.
|
||||
*/
|
||||
suspend fun createGroup(nostrGroupId: HexKey): HexKey {
|
||||
suspend fun createGroup(
|
||||
nostrGroupId: HexKey,
|
||||
initialMetadata: MarmotGroupData? = null,
|
||||
): HexKey {
|
||||
Log.d("MarmotManager") { "createGroup($nostrGroupId): by ${signer.pubKey.take(8)}…" }
|
||||
val identity = signer.pubKey.hexToByteArray()
|
||||
groupManager.createGroup(nostrGroupId, identity)
|
||||
val extras = initialMetadata?.let { listOf(it.toExtension()) } ?: emptyList()
|
||||
groupManager.createGroup(nostrGroupId, identity, initialExtensions = extras)
|
||||
subscriptionManager.subscribeGroup(nostrGroupId)
|
||||
Log.d("MarmotManager") { "createGroup($nostrGroupId): persisted and subscribed" }
|
||||
return nostrGroupId
|
||||
@@ -346,13 +359,24 @@ class MarmotManager(
|
||||
/**
|
||||
* Update group metadata (name, description, etc.) via a GroupContextExtensions proposal.
|
||||
* Creates a GCE proposal, commits it, and returns the commit event to publish.
|
||||
*
|
||||
* RFC 9420 §12.1.7: a GroupContextExtensions proposal REPLACES the entire
|
||||
* extension list in GroupContext. Peers (notably mdk-core / whitenoise-rs)
|
||||
* reject welcomes whose GroupContext is missing the required-capabilities
|
||||
* extension, and strip metadata from groups whose context lacks the
|
||||
* MarmotGroupData extension. We therefore preserve every other existing
|
||||
* extension and overwrite only the slot we actually want to update.
|
||||
*/
|
||||
suspend fun updateGroupMetadata(
|
||||
nostrGroupId: HexKey,
|
||||
metadata: MarmotGroupData,
|
||||
): OutboundGroupEvent {
|
||||
val commitResult =
|
||||
groupManager.updateGroupExtensions(nostrGroupId, listOf(metadata.toExtension()))
|
||||
val group =
|
||||
groupManager.getGroup(nostrGroupId)
|
||||
?: throw IllegalStateException("Not a member of group $nostrGroupId")
|
||||
val preserved = group.extensions.filter { it.extensionType != MarmotGroupData.EXTENSION_ID_INT }
|
||||
val merged = preserved + metadata.toExtension()
|
||||
val commitResult = groupManager.updateGroupExtensions(nostrGroupId, merged)
|
||||
val commitEvent =
|
||||
outboundProcessor.buildCommitEvent(
|
||||
nostrGroupId = nostrGroupId,
|
||||
@@ -480,6 +504,18 @@ class MarmotManager(
|
||||
*/
|
||||
fun groupEpoch(nostrGroupId: HexKey): Long? = groupManager.getGroup(nostrGroupId)?.epoch
|
||||
|
||||
/**
|
||||
* Hex-encoded MLS group id for the group keyed by [nostrGroupId], or null if
|
||||
* that group is not locally known.
|
||||
*
|
||||
* Interop note: whitenoise-rs (and every mdk consumer) indexes groups by the
|
||||
* MLS GroupContext's groupId, NOT the MIP-01 nostr_group_id that we use as
|
||||
* the primary key internally. When a harness or external caller needs to
|
||||
* cross-reference a group with another client (e.g. the interop harness
|
||||
* calling `wn messages list <mls_id>`), it needs this translation.
|
||||
*/
|
||||
fun mlsGroupIdHex(nostrGroupId: HexKey): HexKey? = groupManager.getGroup(nostrGroupId)?.groupId?.toHexKey()
|
||||
|
||||
/**
|
||||
* Resolve the MLS leaf index for a member by Nostr pubkey, or null if that
|
||||
* pubkey isn't currently in the group.
|
||||
|
||||
Reference in New Issue
Block a user