nick/amethyst
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

fix(marmot,cli,interop): interop-compatible Marmot flows and harness correctness

Browse Source 
Five protocol-level fixes and a batch of harness correctness fixes to get
the headless Marmot/Whitenoise interop harness from 1/13 to 5/13 passing
cleanly, with the remaining failures all rooted in wn's per-account
serial event-processor retry backoff (which drops undecryptable
pre-membership commits after several minutes) rather than amy behaviour.

quartz + commons
----------------
* MarmotGroupData: hold CURRENT_VERSION at 2. mdk-core (the Rust MLS
  engine used by whitenoise-rs) strict-rejects v3 payloads with
  `ExtensionFormatError("Trailing bytes in NostrGroupDataExtension")`
  — our v3 welcomes and GCE commits never apply, so every cross-client
  group flow dies at welcome processing. We still parse v3 happily on
  the way in; we just don't emit it until mdk publishes the
  forward-compat fix MIP-01 mandates.
* MarmotManager.updateGroupMetadata: MERGE extensions instead of
  REPLACING. RFC 9420 §12.1.7 says GCE proposals blow away the old
  extension list; callers that pass only [marmot_group_data] dropped
  [required_capabilities], which peers then reject. Preserve every slot
  except the one we're updating.
* MarmotManager.createGroup: new optional `initialMetadata` parameter
  that bakes MarmotGroupData into epoch-0 GroupContext.extensions
  directly. Without it, creators had to publish a pre-membership
  "bootstrap" commit that no later joiner could decrypt — each such
  peer then burned their retry budget on an undecryptable kind:445
  before seeing the real state. Threaded through MlsGroup.create /
  MlsGroupManager.createGroup.
* MarmotManager.mlsGroupIdHex: new translation helper so any code
  juggling the MIP-01 nostr_group_id (what amy indexes on) and the
  MLS GroupContext groupId (what mdk indexes on) can cross-reference
  them without reaching into MlsGroupManager directly.

amethyst module
---------------
* Account.leaveMarmotGroup: self-demote before SelfRemove per MIP-01,
  and promote a surviving member to admin first if the caller is the
  sole admin (otherwise we'd throw "admin depletion"). Matches the
  cli/GroupMembershipCommands.leave flow.

cli (amy)
---------
* Context.syncIncoming: don't advance `giftWrapSince` on empty polls
  (so the first-ever sync doesn't bump the cursor past every
  past-timestamped wrap we've ever been sent), subtract 2 days lookback
  when filtering (NIP-59 randomWithTwoDays gift wraps can have any
  createdAt in the last 48h), and only advance `groupSince` for groups
  we actually received events for.
* Context.syncIncoming: after ingest, if any Welcome consumed a
  KeyPackage, rotate and publish a fresh one immediately. MIP-00
  requires this — a KP can only be welcomed once and leaving the
  consumed one on relays just means later senders invite us with a
  bundle we no longer have private keys for.
* Context.resolveGroupId: accept either nostr_group_id (amy's primary
  key) or the MLS GroupContext groupId (what wn emits) on every verb
  that takes a group id. Wired through GroupAdd/Remove/Leave/Metadata
  /Read, Message send/list, and all the await* verbs so harness
  scripts never have to juggle both forms for a single group.
* GroupCreateCommand: bake initial metadata into epoch 0 (see the
  quartz change above). Dropped the now-redundant bootstrap commit
  publish and tightened the JSON output to include `mls_group_id`.
* GroupMembershipCommands.leave: self-demote admin before SelfRemove;
  promote an heir if we're the only admin, otherwise the GCE would
  deplete admins and the leave aborts.
* MarmotIngest.ingestGiftWrap: unwrap the sealed-rumor layer. NIP-59
  wrap is gift-wrap(kind:1059) → seal(kind:13) → rumor; the old code
  only unwrapped once and then checked `inner.kind == 444`, which is
  always false because inner is actually the seal. Unseal once more
  before the Welcome check. This single fix is what unsticks every
  amy-side Welcome ingestion.

wn harness patches + scripts
----------------------------
* whitenoise-defaults-env.patch: honour $WHITENOISE_DISCOVERY_RELAYS in
  `Relay::defaults()` (release builds otherwise bake damus.io / primal
  / nos.lol into every new account's NIP-65 / Inbox / KeyPackage
  lists, which breaks publishing and prevents the inbox subscription
  plane from ever reaching an operational state in a sandbox).
* setup.sh: sleep 2s after amy's initial kind:30443 publish so
  nostr-rs-relay has a chance to fsync before wn's first targeted
  discovery query. Without it wn's `keys check` races the relay's
  WAL flush and intermittently returns NotFound.
* lib.sh: peel wn's `{"result": …}` wrapper in `jq_group_id`,
  `wait_for_invite`, `wait_for_message`, `wait_for_member`. Post-v0.2
  wn `--json` output nests everything under `.result` (and
  `groups invites[]` nests further under `.group.mls_group_id`) —
  these helpers were still pattern-matching on the flat shape, so
  they returned empty strings for a perfectly good response.
* tests-{create,manage,extras}.sh: track both group IDs per test
  (amy's nostr + wn's MLS), pass each CLI the id it understands, and
  bump the post-commit wait timeouts to 90–120s so wn's exponential
  retry backoff has time to work through the pre-membership commits
  it can't decrypt and get to the ones it can.

https://claude.ai/code/session_016kAxdp6ubB5CnF9URhCEzP
This commit is contained in:
Claude
2026-04-22 00:28:45 +00:00
parent ae1549b884
commit f08b010f50
20 changed files with 422 additions and 126 deletions
Download Patch File Download Diff File Expand all files Collapse all files
amethyst/src/main/java/com/vitorpamplona/amethyst/model/Account.kt
+29
View File
@@ -2260,6 +2260,14 @@ class Account(
/**
* Leave a Marmot MLS group.
* Publishes the SelfRemove proposal and removes local state.
*
* MIP-01/MIP-03: admins MUST first publish a GroupContextExtensions
* commit dropping themselves from `admin_pubkeys` before issuing a
* SelfRemove proposal. Without that, [MlsGroup.selfRemove] throws
* `IllegalStateException("Admin must self-demote via GroupContextExtensions
* before SelfRemove (MIP-01)")` and the leave aborts. Demote commit and
* SelfRemove proposal both go to the same group relays, demote first so
* peers apply it before they see the SelfRemove.
*/
suspend fun leaveMarmotGroup(
nostrGroupId: HexKey,
@@ -2268,6 +2276,27 @@ class Account(
val manager = marmotManager ?: return
if (!isWriteable()) return
val metadata = manager.groupMetadata(nostrGroupId)
if (metadata != null && metadata.adminPubkeys.contains(signer.pubKey)) {
val remaining = metadata.adminPubkeys.filter { it != signer.pubKey }.toMutableList()
// MIP-03 also rejects any GCE commit that leaves the group with zero
// admins. If we're the only one, promote an arbitrary non-self
// member to admin before stepping down.
if (remaining.isEmpty()) {
val heir =
manager
.memberPubkeys(nostrGroupId)
.map { it.pubkey }
.firstOrNull { it != signer.pubKey }
if (heir != null) remaining.add(heir)
}
if (remaining.isNotEmpty()) {
val demoted = metadata.copy(adminPubkeys = remaining)
val demoteCommit = manager.updateGroupMetadata(nostrGroupId, demoted)
client.publish(demoteCommit.signedEvent, groupRelays)
}
}
val outbound = manager.leaveGroup(nostrGroupId)
client.publish(outbound.signedEvent, groupRelays)
}
cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/Context.kt
+76 -7
View File
@@ -219,14 +219,26 @@ class Context(
* - kind:445 group events per active group → feed into inbound processor
*
* Incrementally advances the `since` cursors in [state] so the next run
* only asks relays for newer events.
* only asks relays for newer events. Two wrinkles:
*
* 1. NIP-59 gift wraps are published with a random-past `created_at`
* (see [com.vitorpamplona.quartz.utils.TimeUtils.randomWithTwoDays])
* so a newly-published wrap can trivially have `created_at` earlier
* than the last cursor we saw. To avoid silently dropping such wraps
* we always subtract a 2-day lookback window from the gift-wrap
* `since`, and dedup is handled inside [MarmotInboundProcessor].
* 2. We only advance the on-disk cursor when events actually arrive.
* Snapping an empty sync up to "now" on the first invocation would
* make every later `since` query skip any past-dated wrap or 445.
*/
suspend fun syncIncoming(timeoutMs: Long = 8_000) {
val inbox = inboxRelays().ifEmpty { anyRelays() }
val gwSince = state.giftWrapSince
val gwFilterSince =
gwSince?.let { (it - GIFT_WRAP_LOOKBACK_SECS).coerceAtLeast(0L) }
val gwFilter =
if (gwSince != null) {
MarmotFilters.giftWrapsForUserSince(identity.pubKeyHex, gwSince)
if (gwFilterSince != null) {
MarmotFilters.giftWrapsForUserSince(identity.pubKeyHex, gwFilterSince)
} else {
MarmotFilters.giftWrapsForUser(identity.pubKeyHex)
}
@@ -255,10 +267,11 @@ class Context(
if (filterMap.isEmpty()) return
val events = drain(filterMap, timeoutMs)
val now = System.currentTimeMillis() / 1000
var maxGwSeen = gwSince ?: 0L
val maxGroupSeen = perGroupFilters.keys.associateWith { state.groupSince[it] ?: 0L }.toMutableMap()
var sawGiftWrap = false
val sawGroupEvent = mutableSetOf<HexKey>()
for ((relay, event) in events) {
// All the MLS/NIP-59 decryption + persistence lives in MarmotIngest —
@@ -268,21 +281,70 @@ class Context(
when (event.kind) {
GiftWrapEvent.KIND -> {
sawGiftWrap = true
if (event.createdAt > maxGwSeen) maxGwSeen = event.createdAt
}
GroupEvent.KIND -> {
val gid = (event as? GroupEvent)?.groupId() ?: continue
sawGroupEvent.add(gid)
val prev = maxGroupSeen[gid] ?: 0L
if (event.createdAt > prev) maxGroupSeen[gid] = event.createdAt
}
}
}
state.giftWrapSince = if (maxGwSeen > 0) maxGwSeen else now
for ((gid, seen) in maxGroupSeen) {
state.groupSince[gid] = if (seen > 0) seen else now
if (sawGiftWrap && maxGwSeen > 0) {
state.giftWrapSince = maxGwSeen
}
for (gid in sawGroupEvent) {
val seen = maxGroupSeen[gid] ?: continue
if (seen > 0) state.groupSince[gid] = seen
}
// If any welcome we processed consumed a KeyPackage, MIP-00 requires
// us to immediately publish a replacement (a KP can only be used for
// ONE welcome; leaving the old one on relays lets a second sender
// invite us with a bundle we no longer have private keys for). The
// Amethyst UI handles this via its own rotation scheduler; the CLI
// has no scheduler, so we rotate inline right after sync.
if (marmot.needsKeyPackageRotation()) {
try {
val kpRelays = keyPackageRelays().ifEmpty { outboxRelays() }.ifEmpty { anyRelays() }
if (kpRelays.isNotEmpty()) {
val rotated = marmot.rotateConsumedKeyPackages(kpRelays.toList())
for (event in rotated) {
publish(event, kpRelays)
System.err.println("[cli] rotated KeyPackage → ${event.id.take(8)} on ${kpRelays.size} relay(s)")
}
}
} catch (e: Exception) {
System.err.println("[cli] key-package rotation failed: ${e.message}")
}
}
}
/**
* Resolve a group identifier given on the CLI to the nostr_group_id that
* amy's [MarmotManager] indexes on.
*
* amy internally keys everything off MIP-01's `nostr_group_id`. whitenoise
* (and every other mdk consumer) keys off the MLS `GroupContext.groupId` —
* a separate 32-byte random value stamped at group creation. Cross-client
* scripts therefore wind up juggling both ids, and it's very easy to pass
* the wrong one to amy. Rather than make every caller translate, we accept
* either format and resolve here:
* 1. If the input is an active nostr_group_id, use it unchanged.
* 2. Otherwise scan active groups for one whose MLS groupId matches.
* 3. Otherwise return the input unchanged (so the caller still gets a
* sensible `not_member` response rather than a silent mismatch).
*/
fun resolveGroupId(input: HexKey): HexKey {
if (marmot.isMember(input)) return input
val normalized = input.lowercase()
return marmot.activeGroupIds().firstOrNull { nostrId ->
marmot.mlsGroupIdHex(nostrId)?.lowercase() == normalized
} ?: input
}
fun marmotGroupRelays(nostrGroupId: HexKey): Set<NormalizedRelayUrl> {
@@ -303,6 +365,13 @@ class Context(
}
companion object {
/**
* Lookback applied to the gift-wrap `since` filter to compensate for
* NIP-59's randomised-past `created_at`. 2 days matches
* [com.vitorpamplona.quartz.utils.TimeUtils.randomWithTwoDays].
*/
private const val GIFT_WRAP_LOOKBACK_SECS: Long = 2L * 24 * 60 * 60
/** Build a Context but require an identity to already exist — most commands can't run without one. */
fun open(dataDir: DataDir): Context {
val identity =
cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/AwaitCommands.kt
+6 -5
View File
@@ -108,6 +108,7 @@ object AwaitCommands {
Json.writeLine(
mapOf(
"group_id" to match,
"mls_group_id" to ctx.marmot.mlsGroupIdHex(match),
"name" to (ctx.marmot.groupMetadata(match)?.name ?: ""),
"epoch" to ctx.marmot.groupEpoch(match),
),
@@ -127,7 +128,7 @@ object AwaitCommands {
rest: Array<String>,
): Int =
pollCondition(dataDir, rest, "await member <gid> <npub>", targetIdx = 1) { ctx, rawArgs ->
val gid = rawArgs[0]
val gid = ctx.resolveGroupId(rawArgs[0])
val target = ctx.requireUserHex(rawArgs[1])
if (!ctx.marmot.isMember(gid)) {
null
@@ -143,7 +144,7 @@ object AwaitCommands {
rest: Array<String>,
): Int =
pollCondition(dataDir, rest, "await admin <gid> <npub>", targetIdx = 1) { ctx, rawArgs ->
val gid = rawArgs[0]
val gid = ctx.resolveGroupId(rawArgs[0])
val target = ctx.requireUserHex(rawArgs[1])
if (!ctx.marmot.isMember(gid)) {
null
@@ -163,13 +164,13 @@ object AwaitCommands {
rest: Array<String>,
): Int {
if (rest.isEmpty()) return Json.error("bad_args", "await rename <gid> --name <name>")
val gid = rest[0]
val args = Args(rest.drop(1).toTypedArray())
val wantedName = args.requireFlag("name")
val timeoutSecs = args.longFlag("timeout", 30)
val ctx = Context.open(dataDir)
try {
ctx.prepare()
val gid = ctx.resolveGroupId(rest[0])
val deadline = System.currentTimeMillis() + timeoutSecs * 1000
while (System.currentTimeMillis() < deadline) {
ctx.syncIncoming(timeoutMs = 3_000)
@@ -191,13 +192,13 @@ object AwaitCommands {
rest: Array<String>,
): Int {
if (rest.isEmpty()) return Json.error("bad_args", "await epoch <gid> --min N")
val gid = rest[0]
val args = Args(rest.drop(1).toTypedArray())
val min = args.longFlag("min", 1)
val timeoutSecs = args.longFlag("timeout", 30)
val ctx = Context.open(dataDir)
try {
ctx.prepare()
val gid = ctx.resolveGroupId(rest[0])
val deadline = System.currentTimeMillis() + timeoutSecs * 1000
while (System.currentTimeMillis() < deadline) {
ctx.syncIncoming(timeoutMs = 3_000)
@@ -219,13 +220,13 @@ object AwaitCommands {
rest: Array<String>,
): Int {
if (rest.isEmpty()) return Json.error("bad_args", "await message <gid> --match STRING")
val gid = rest[0]
val args = Args(rest.drop(1).toTypedArray())
val needle = args.requireFlag("match")
val timeoutSecs = args.longFlag("timeout", 30)
val ctx = Context.open(dataDir)
try {
ctx.prepare()
val gid = ctx.resolveGroupId(rest[0])
val deadline = System.currentTimeMillis() + timeoutSecs * 1000
while (System.currentTimeMillis() < deadline) {
ctx.syncIncoming(timeoutMs = 3_000)
cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/GroupAddMemberCommand.kt
+1 -1
View File
@@ -37,10 +37,10 @@ object GroupAddMemberCommand {
rest: Array<String>,
): Int {
if (rest.size < 2) return Json.error("bad_args", "group add <group_id> <npub> [<npub> ...]")
val gid = rest[0]
val ctx = Context.open(dataDir)
try {
ctx.prepare()
val gid = ctx.resolveGroupId(rest[0])
ctx.syncIncoming()
if (!ctx.marmot.isMember(gid)) return Json.error("not_member", gid)
cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/GroupCreateCommand.kt
+8 -10
View File
@@ -40,9 +40,12 @@ object GroupCreateCommand {
ctx.prepare()
val gid = RandomInstance.bytes(32).toHexKey()
ctx.marmot.createGroup(gid)
// Stamp initial metadata via the shared factory so UI + CLI stay byte-identical.
// Stamp initial metadata via the shared factory so UI + CLI stay
// byte-identical. Bake the MarmotGroupData extension into the
// epoch-0 GroupContext directly (see `MarmotManager.createGroup`)
// so later invitees receive a pre-populated group from the
// welcome and never have to chase an undecryptable bootstrap
// commit that predates their membership.
val outboxUrls = ctx.outboxRelays().map { it.url }
val metadata =
MarmotGroupData.bootstrap(
@@ -51,19 +54,14 @@ object GroupCreateCommand {
outboxRelays = outboxUrls,
name = name,
)
val commit = ctx.marmot.updateGroupMetadata(gid, metadata)
// Group relays == what the metadata carries, which on first commit is our outbox.
val targets = ctx.outboxRelays()
val ack = ctx.publish(commit.signedEvent, targets)
ctx.marmot.createGroup(gid, initialMetadata = metadata)
Json.writeLine(
mapOf(
"group_id" to gid,
"mls_group_id" to ctx.marmot.mlsGroupIdHex(gid),
"name" to name,
"epoch" to ctx.marmot.groupEpoch(gid),
"commit_event_id" to commit.signedEvent.id,
"published_to" to ack.filterValues { it }.keys.map { it.url },
),
)
return 0
cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/GroupMembershipCommands.kt
+30 -2
View File
@@ -30,10 +30,10 @@ object GroupMembershipCommands {
rest: Array<String>,
): Int {
if (rest.size < 2) return Json.error("bad_args", "group remove <gid> <npub>")
val gid = rest[0]
val ctx = Context.open(dataDir)
try {
ctx.prepare()
val gid = ctx.resolveGroupId(rest[0])
val target = ctx.requireUserHex(rest[1])
ctx.syncIncoming()
if (!ctx.marmot.isMember(gid)) return Json.error("not_member", gid)
@@ -66,18 +66,46 @@ object GroupMembershipCommands {
rest: Array<String>,
): Int {
if (rest.isEmpty()) return Json.error("bad_args", "group leave <gid>")
val gid = rest[0]
val ctx = Context.open(dataDir)
try {
ctx.prepare()
val gid = ctx.resolveGroupId(rest[0])
if (!ctx.marmot.isMember(gid)) return Json.error("not_member", gid)
val targets = ctx.marmotGroupRelays(gid).ifEmpty { ctx.outboxRelays() }
// MIP-01/MIP-03: members listed in `admin_pubkeys` MUST NOT issue
// a SelfRemove proposal before first publishing a GCE that drops
// themselves from the admin list (MlsGroup.selfRemove enforces
// this with `check(!isLocalAdmin())`), and that same GCE MUST NOT
// leave the group with zero admins (admin depletion). If we're
// the only admin, hand admin to another member first.
val demoteEventId: String? =
ctx.marmot.groupMetadata(gid)?.let { metadata ->
if (!metadata.adminPubkeys.contains(ctx.identity.pubKeyHex)) return@let null
val newAdmins = metadata.adminPubkeys.filter { it != ctx.identity.pubKeyHex }.toMutableList()
if (newAdmins.isEmpty()) {
val heir =
ctx.marmot
.memberPubkeys(gid)
.map { it.pubkey }
.firstOrNull { it != ctx.identity.pubKeyHex }
?: return@let null // solo group — skip demote, let MLS state cleanup handle it
newAdmins.add(heir)
}
val demoted = metadata.copy(adminPubkeys = newAdmins)
val demoteCommit = ctx.marmot.updateGroupMetadata(gid, demoted)
ctx.publish(demoteCommit.signedEvent, targets)
demoteCommit.signedEvent.id
}
val outbound = ctx.marmot.leaveGroup(gid)
val ack = ctx.publish(outbound.signedEvent, targets)
Json.writeLine(
mapOf(
"group_id" to gid,
"self_demote_event_id" to demoteEventId,
"proposal_event_id" to outbound.signedEvent.id,
"published_to" to ack.filterValues { it }.keys.map { it.url },
),
cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/GroupMetadataCommands.kt
+2 -1
View File
@@ -66,12 +66,13 @@ object GroupMetadataCommands {
private suspend fun edit(
dataDir: DataDir,
gid: HexKey,
rawGid: HexKey,
mutate: suspend (Context, MarmotGroupData) -> MarmotGroupData,
): Int {
val ctx = Context.open(dataDir)
try {
ctx.prepare()
val gid = ctx.resolveGroupId(rawGid)
ctx.syncIncoming()
if (!ctx.marmot.isMember(gid)) return Json.error("not_member", gid)
val outboxUrls = ctx.outboxRelays().map { it.url }
cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/GroupReadCommands.kt
+4 -3
View File
@@ -56,10 +56,10 @@ object GroupReadCommands {
rest: Array<String>,
): Int {
if (rest.isEmpty()) return Json.error("bad_args", "group show <group_id>")
val gid = rest[0]
val ctx = Context.open(dataDir)
try {
ctx.prepare()
val gid = ctx.resolveGroupId(rest[0])
ctx.syncIncoming()
if (!ctx.marmot.isMember(gid)) return Json.error("not_member", gid)
val meta = ctx.marmot.groupMetadata(gid)
@@ -70,6 +70,7 @@ object GroupReadCommands {
Json.writeLine(
mapOf(
"group_id" to gid,
"mls_group_id" to ctx.marmot.mlsGroupIdHex(gid),
"name" to (meta?.name ?: ""),
"description" to (meta?.description ?: ""),
"epoch" to ctx.marmot.groupEpoch(gid),
@@ -90,10 +91,10 @@ object GroupReadCommands {
rest: Array<String>,
): Int {
if (rest.isEmpty()) return Json.error("bad_args", "group members <group_id>")
val gid = rest[0]
val ctx = Context.open(dataDir)
try {
ctx.prepare()
val gid = ctx.resolveGroupId(rest[0])
ctx.syncIncoming()
if (!ctx.marmot.isMember(gid)) return Json.error("not_member", gid)
val members =
@@ -112,10 +113,10 @@ object GroupReadCommands {
rest: Array<String>,
): Int {
if (rest.isEmpty()) return Json.error("bad_args", "group admins <group_id>")
val gid = rest[0]
val ctx = Context.open(dataDir)
try {
ctx.prepare()
val gid = ctx.resolveGroupId(rest[0])
ctx.syncIncoming()
if (!ctx.marmot.isMember(gid)) return Json.error("not_member", gid)
val m = ctx.marmot.groupMetadata(gid)
cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/MessageCommands.kt
+2 -2
View File
@@ -45,11 +45,11 @@ object MessageCommands {
rest: Array<String>,
): Int {
if (rest.size < 2) return Json.error("bad_args", "message send <gid> <text>")
val gid = rest[0]
val text = rest[1]
val ctx = Context.open(dataDir)
try {
ctx.prepare()
val gid = ctx.resolveGroupId(rest[0])
ctx.syncIncoming()
if (!ctx.marmot.isMember(gid)) return Json.error("not_member", gid)
@@ -77,12 +77,12 @@ object MessageCommands {
rest: Array<String>,
): Int {
if (rest.isEmpty()) return Json.error("bad_args", "message list <gid>")
val gid = rest[0]
val args = Args(rest.drop(1).toTypedArray())
val limit = args.intFlag("limit", Int.MAX_VALUE)
val ctx = Context.open(dataDir)
try {
ctx.prepare()
val gid = ctx.resolveGroupId(rest[0])
ctx.syncIncoming()
if (!ctx.marmot.isMember(gid)) return Json.error("not_member", gid)
commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/marmot/MarmotIngest.kt
+16 -3
View File
@@ -27,6 +27,7 @@ import com.vitorpamplona.quartz.marmot.mip02Welcome.WelcomeEvent
import com.vitorpamplona.quartz.marmot.mip03GroupMessages.GroupEvent
import com.vitorpamplona.quartz.nip01Core.core.Event
import com.vitorpamplona.quartz.nip01Core.core.HexKey
import com.vitorpamplona.quartz.nip59Giftwrap.seals.SealedRumorEvent
import com.vitorpamplona.quartz.nip59Giftwrap.wraps.GiftWrapEvent
/**
@@ -95,11 +96,23 @@ suspend fun MarmotManager.ingest(event: Event): MarmotIngestResult =
private suspend fun MarmotManager.ingestGiftWrap(wrap: GiftWrapEvent): MarmotIngestResult =
try {
val inner = wrap.unwrapOrNull(signer) ?: return MarmotIngestResult.Ignored
if (!MarmotInboundProcessor.isWelcomeEvent(inner) || inner !is WelcomeEvent) {
// NIP-59 wraps contain TWO encryption layers:
// kind:1059 gift wrap → kind:13 sealed rumor → the rumor itself.
// `GiftWrapEvent.unwrapOrNull` only peels the outer layer; when the
// result is a [SealedRumorEvent] we must unseal it to reach the
// kind:444 Welcome rumor. The old code checked `isWelcomeEvent` on
// the seal (kind:13) and always took the Ignored branch, which is
// why every inbound Welcome was silently dropped by the CLI and by
// any non-Amethyst consumer.
val rumor =
when (val inner = wrap.unwrapOrNull(signer) ?: return MarmotIngestResult.Ignored) {
is SealedRumorEvent -> inner.unsealOrNull(signer) ?: return MarmotIngestResult.Ignored
else -> inner
}
if (!MarmotInboundProcessor.isWelcomeEvent(rumor) || rumor !is WelcomeEvent) {
return MarmotIngestResult.Ignored
}
when (val result = processWelcome(inner, inner.nostrGroupId())) {
when (val result = processWelcome(rumor, rumor.nostrGroupId())) {
is WelcomeResult.Joined -> {
MarmotIngestResult.JoinedGroup(
nostrGroupId = result.nostrGroupId,
commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/marmot/MarmotManager.kt
+40 -4
View File
@@ -262,11 +262,24 @@ class MarmotManager(
/**
* Create a new MLS group.
*
* When [initialMetadata] is non-null it is baked into epoch 0's
* GroupContext.extensions. Later joiners' welcomes therefore carry the
* group name / admin list / relays from the get-go and no separate
* "bootstrap commit" needs to be published. The bootstrap-commit path
* works in theory, but it produces a kind:445 encrypted with epoch 0's
* exporter secret that no post-membership peer (amethyst or wn) has,
* so each such peer wastes their commit-retry budget on an
* undecryptable event before processing the real state.
*/
suspend fun createGroup(nostrGroupId: HexKey): HexKey {
suspend fun createGroup(
nostrGroupId: HexKey,
initialMetadata: MarmotGroupData? = null,
): HexKey {
Log.d("MarmotManager") { "createGroup($nostrGroupId): by ${signer.pubKey.take(8)}…" }
val identity = signer.pubKey.hexToByteArray()
groupManager.createGroup(nostrGroupId, identity)
val extras = initialMetadata?.let { listOf(it.toExtension()) } ?: emptyList()
groupManager.createGroup(nostrGroupId, identity, initialExtensions = extras)
subscriptionManager.subscribeGroup(nostrGroupId)
Log.d("MarmotManager") { "createGroup($nostrGroupId): persisted and subscribed" }
return nostrGroupId
@@ -346,13 +359,24 @@ class MarmotManager(
/**
* Update group metadata (name, description, etc.) via a GroupContextExtensions proposal.
* Creates a GCE proposal, commits it, and returns the commit event to publish.
*
* RFC 9420 §12.1.7: a GroupContextExtensions proposal REPLACES the entire
* extension list in GroupContext. Peers (notably mdk-core / whitenoise-rs)
* reject welcomes whose GroupContext is missing the required-capabilities
* extension, and strip metadata from groups whose context lacks the
* MarmotGroupData extension. We therefore preserve every other existing
* extension and overwrite only the slot we actually want to update.
*/
suspend fun updateGroupMetadata(
nostrGroupId: HexKey,
metadata: MarmotGroupData,
): OutboundGroupEvent {
val commitResult =
groupManager.updateGroupExtensions(nostrGroupId, listOf(metadata.toExtension()))
val group =
groupManager.getGroup(nostrGroupId)
?: throw IllegalStateException("Not a member of group $nostrGroupId")
val preserved = group.extensions.filter { it.extensionType != MarmotGroupData.EXTENSION_ID_INT }
val merged = preserved + metadata.toExtension()
val commitResult = groupManager.updateGroupExtensions(nostrGroupId, merged)
val commitEvent =
outboundProcessor.buildCommitEvent(
nostrGroupId = nostrGroupId,
@@ -480,6 +504,18 @@ class MarmotManager(
*/
fun groupEpoch(nostrGroupId: HexKey): Long? = groupManager.getGroup(nostrGroupId)?.epoch
/**
* Hex-encoded MLS group id for the group keyed by [nostrGroupId], or null if
* that group is not locally known.
*
* Interop note: whitenoise-rs (and every mdk consumer) indexes groups by the
* MLS GroupContext's groupId, NOT the MIP-01 nostr_group_id that we use as
* the primary key internally. When a harness or external caller needs to
* cross-reference a group with another client (e.g. the interop harness
* calling `wn messages list <mls_id>`), it needs this translation.
*/
fun mlsGroupIdHex(nostrGroupId: HexKey): HexKey? = groupManager.getGroup(nostrGroupId)?.groupId?.toHexKey()
/**
* Resolve the MLS leaf index for a member by Nostr pubkey, or null if that
* pubkey isn't currently in the group.
quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/marmot/mip01Groups/MarmotGroupData.kt
+16 -1
View File
@@ -191,7 +191,22 @@ data class MarmotGroupData(
fun toExtension(): Extension = Extension(EXTENSION_ID_INT, encodeTls())
companion object {
const val CURRENT_VERSION = 3
/**
* Version we emit for freshly created groups and fresh GCE commits.
*
* Held at 2 (not 3) because the Rust mdk-core MLS engine used by
* whitenoise-rs (the only other shipping Marmot client today) is
* stricter than MIP-01's "ignore trailing bytes for forward
* compatibility" rule — it rejects v3 payloads with
* `ExtensionFormatError("Trailing bytes in NostrGroupDataExtension")`,
* which means our v3 welcomes/commits never get applied and every
* cross-client group flow breaks. We still PARSE v3 happily via
* [decodeTls] (any peer that sends us a v3 group will round-trip),
* but we don't create them until mdk-core catches up.
*
* Bump back to 3 once mdk publishes the forward-compat fix.
*/
const val CURRENT_VERSION = 2
/** Versions this implementation understands. v0 is reserved/invalid per MIP-01. */
val SUPPORTED_VERSIONS: Set<Int> = setOf(1, 2, 3)
quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/marmot/mls/group/MlsGroup.kt
+7 -1
View File
@@ -1929,6 +1929,7 @@ class MlsGroup private constructor(
fun create(
identity: ByteArray,
signingKey: ByteArray? = null,
initialExtensions: List<com.vitorpamplona.quartz.marmot.mls.tree.Extension> = emptyList(),
): MlsGroup {
val sigKp =
signingKey?.let { key ->
@@ -1953,13 +1954,18 @@ class MlsGroup private constructor(
tree.setLeaf(0, leafNode)
val treeHash = tree.treeHash()
// Start with required_capabilities + whatever the caller wants to
// bake into epoch 0 (e.g. the MIP-01 MarmotGroupData extension so
// new peers who join later can see the group name without first
// decrypting a pre-membership bootstrap commit — see MIP-03).
val baseExtensions = listOf(buildMarmotRequiredCapabilitiesExtension())
val groupContext =
GroupContext(
groupId = groupId,
epoch = 0,
treeHash = treeHash,
confirmedTranscriptHash = ByteArray(0),
extensions = listOf(buildMarmotRequiredCapabilitiesExtension()),
extensions = baseExtensions + initialExtensions,
)
// Initial key schedule with zero secrets
quartz/src/commonMain/kotlin/com/vitorpamplona/quartz/marmot/mls/group/MlsGroupManager.kt
+2 -1
View File
@@ -174,10 +174,11 @@ class MlsGroupManager(
nostrGroupId: HexKey,
identity: ByteArray,
signingKey: ByteArray? = null,
initialExtensions: List<com.vitorpamplona.quartz.marmot.mls.tree.Extension> = emptyList(),
): MlsGroup =
mutex.withLock {
Log.d(TAG) { "createGroup($nostrGroupId): creating new MLS group" }
val group = MlsGroup.create(identity, signingKey)
val group = MlsGroup.create(identity, signingKey, initialExtensions)
groups[nostrGroupId] = group
persistGroup(nostrGroupId)
Log.d(TAG) { "createGroup($nostrGroupId): done, in-memory group count=${groups.size}" }
tools/marmot-interop/headless/patches/whitenoise-defaults-env.patch
+24
View File
@@ -0,0 +1,24 @@
--- a/src/whitenoise/relays.rs
+++ b/src/whitenoise/relays.rs
@@ -98,6 +98,21 @@
}
pub(crate) fn defaults() -> Vec<Relay> {
+ // marmot-interop-headless patch: honour $WHITENOISE_DISCOVERY_RELAYS
+ // (comma-separated list) when present so newly created accounts only
+ // ever get our loopback relay baked into their NIP-65 / inbox /
+ // key-package lists. Without this override, `create-identity` stamps
+ // the hard-coded public set into the account's relay lists, and every
+ // later activate / publish burns connection budget on unreachable
+ // sockets — enough to break inbox-plane activation and drop kind:1059.
+ if let Ok(from_env) = std::env::var("WHITENOISE_DISCOVERY_RELAYS") {
+ let parsed: Vec<Relay> = from_env
+ .split(',').map(str::trim).filter(|s| !s.is_empty())
+ .filter_map(|u| RelayUrl::parse(u).ok())
+ .map(|url| Relay::new(&url))
+ .collect();
+ if !parsed.is_empty() { return parsed; }
+ }
let urls: &[&str] = if cfg!(debug_assertions) {
&["ws://localhost:8080", "ws://localhost:7777"]
} else {
tools/marmot-interop/headless/setup.sh
+14 -1
View File
@@ -39,7 +39,7 @@ preflight() {
2>&1 | tee -a "$LOG_FILE"
fi
# Two harness-only patches to wnd so it runs fully offline / in
# Three harness-only patches to wnd so it runs fully offline / in
# sandboxes that block outbound + kernel keyring:
# 1. discovery-env: honour $WHITENOISE_DISCOVERY_RELAYS so we can
# point wnd at our loopback relay instead of the baked-in public
@@ -47,9 +47,17 @@ preflight() {
# 2. mock-keyring: honour $WHITENOISE_MOCK_KEYRING so wnd uses the
# integration-tests mock keyring store when the kernel keyutils
# syscalls are blocked (common in containers / CI).
# 3. defaults-env: reuse the same env var so `Relay::defaults()`
# (what `create-identity` stamps into the new account's NIP-65 /
# inbox / key-package lists) points at the loopback relay too.
# Without it every account wnd creates carries damus.io /
# primal.net / nos.lol, and every later activate / publish burns
# connection budget on unreachable sockets — enough to break the
# account-inbox subscription plane and drop kind:1059 delivery.
local -a patches=(
"whitenoise-discovery-env.patch"
"whitenoise-mock-keyring.patch"
"whitenoise-defaults-env.patch"
)
for name in "${patches[@]}"; do
local marker="$WN_REPO/.headless-patched-${name%.patch}"
@@ -267,4 +275,9 @@ configure_relays() {
step "publishing A's KeyPackage"
amy_a marmot key-package publish >>"$LOG_FILE" 2>&1 || warn "amy marmot key-package publish failed"
# Give nostr-rs-relay a breath to fsync the kind:10002 / 10050 / 30443
# writes and push them out on the discovery subscription so that the
# first `wn keys check` that follows actually sees them instead of
# racing the relay's WAL flush.
sleep 2
}
tools/marmot-interop/headless/tests-create.sh
+47 -23
View File
@@ -33,12 +33,19 @@ test_02_a_creates_group() {
banner "Test 02 — A creates group, invites B"
local id="02 A->B create+invite"
local gid
gid=$(amy_field '.group_id' marmot group create --name "Interop-02") || {
# amy keys groups by the MIP-01 nostr_group_id (`group_id`), wn (via
# mdk-core) keys by the MLS GroupContext groupId (`mls_group_id`). Both
# are 32 random bytes and they are NOT the same. We need both: amy calls
# use `gid`, wn calls use `mls_gid`.
local out gid mls_gid
out=$(amy_json marmot group create --name "Interop-02") || {
record_result "$id" fail "create returned no group_id"; return
}
gid=$(printf '%s' "$out" | jq -r '.group_id')
mls_gid=$(printf '%s' "$out" | jq -r '.mls_group_id')
save_state GROUP_02 "$gid"
info "created group $gid"
save_state GROUP_02_MLS "$mls_gid"
info "created group nostr=$gid mls=$mls_gid"
amy_json marmot group add "$gid" "$B_NPUB" >/dev/null || {
record_result "$id" fail "amy group add failed"; return
@@ -52,16 +59,17 @@ test_02_a_creates_group() {
wn_b groups accept "$b_gid" >/dev/null 2>&1 || true
info "B joined $b_gid"
# A -> B message
# A -> B message. amy takes the nostr id; wait_for_message calls
# `wn messages list` which needs the MLS id.
amy_json marmot message send "$gid" "hello from amethyst" >/dev/null || {
record_result "$id" fail "amy send failed"; return
}
if ! wait_for_message B "$gid" "hello from amethyst" 30; then
if ! wait_for_message B "$mls_gid" "hello from amethyst" 90; then
record_result "$id" fail "B didn't receive A's message"; return
fi
# B -> A message
wn_b messages send "$gid" "hello from wn" >/dev/null 2>&1 || warn "wn send returned nonzero"
wn_b messages send "$mls_gid" "hello from wn" >/dev/null 2>&1 || warn "wn send returned nonzero"
if ! amy_json marmot await message "$gid" --match "hello from wn" --timeout 30 >/dev/null; then
record_result "$id" fail "A didn't receive B's reply"; return
fi
@@ -72,33 +80,41 @@ test_03_b_creates_group() {
banner "Test 03 — B creates group, invites A"
local id="03 B->A create+invite"
local out gid
# `wn groups create` returns the MLS group id (what wn's messages API
# expects). amy indexes by the MIP-01 nostr_group_id, which we only learn
# once A has processed the welcome and we can ask `amy await group` for
# it. Keep them distinct so each CLI gets the id it understands.
local out mls_gid
out=$(wn_b --json groups create "Interop-03" "$A_NPUB" 2>>"$LOG_FILE") || {
record_result "$id" fail "wn groups create failed"; return
}
gid=$(printf '%s' "$out" | jq_group_id)
if [[ -z "$gid" ]]; then
mls_gid=$(printf '%s' "$out" | jq_group_id)
if [[ -z "$mls_gid" ]]; then
record_result "$id" fail "could not parse group_id"; return
fi
save_state GROUP_03 "$gid"
info "group_id: $gid"
save_state GROUP_03_MLS "$mls_gid"
info "mls_group_id: $mls_gid"
# A: poll until it joins.
if ! amy_json marmot await group --name "Interop-03" --timeout 30 >/dev/null; then
# A: poll until it joins, and capture its nostr_group_id.
local a_out a_gid
a_out=$(amy_json marmot await group --name "Interop-03" --timeout 30) || {
record_result "$id" fail "A never joined B's group"; return
fi
}
a_gid=$(printf '%s' "$a_out" | jq -r '.group_id')
save_state GROUP_03 "$a_gid"
info "A joined as nostr_group_id=$a_gid"
# B -> A message
wn_b messages send "$gid" "ping from wn" >/dev/null 2>&1 || true
if ! amy_json marmot await message "$gid" --match "ping from wn" --timeout 30 >/dev/null; then
wn_b messages send "$mls_gid" "ping from wn" >/dev/null 2>&1 || true
if ! amy_json marmot await message "$a_gid" --match "ping from wn" --timeout 30 >/dev/null; then
record_result "$id" fail "A didn't see B's ping"; return
fi
# A -> B reply
amy_json marmot message send "$gid" "pong from amethyst" >/dev/null || {
amy_json marmot message send "$a_gid" "pong from amethyst" >/dev/null || {
record_result "$id" fail "amy send pong failed"; return
}
if wait_for_message B "$gid" "pong from amethyst" 30; then
if wait_for_message B "$mls_gid" "pong from amethyst" 90; then
record_result "$id" pass
else
record_result "$id" fail "B didn't see A's pong"
@@ -112,7 +128,9 @@ test_04_three_member_group() {
wn_c keys publish >/dev/null 2>&1 || true
sleep 3
local gid; gid=$(load_state GROUP_02 || true)
local gid mls_gid
gid=$(load_state GROUP_02 || true)
mls_gid=$(load_state GROUP_02_MLS || true)
if [[ -z "${gid:-}" ]]; then
record_result "$id" skip "no GROUP_02"; return
fi
@@ -131,8 +149,14 @@ test_04_three_member_group() {
amy_json marmot message send "$gid" "hello three-member world" >/dev/null || {
record_result "$id" fail "amy send failed"; return
}
if wait_for_message B "$gid" "hello three-member world" 30 \
&& wait_for_message C "$c_gid" "hello three-member world" 30; then
# wn's event_processor retries undecryptable pre-membership commits with
# exponential backoff (2+4+8+16=~30s), and kind:445 processing is serial
# per-account; a fresh joiner routinely needs ~60s to burn through that
# retry queue before the add-C commit is applied and "hello three-member
# world" decrypts. The 30s we used for the inline A<->B send/recv
# wouldn't clear that.
if wait_for_message B "$mls_gid" "hello three-member world" 90 \
&& wait_for_message C "$c_gid" "hello three-member world" 90; then
record_result "$id" pass
else
record_result "$id" fail "B or C missed A's post-add message"
@@ -166,8 +190,8 @@ test_05_b_adds_a_existing() {
amy_json marmot message send "$gid" "joined from amethyst" >/dev/null || {
record_result "$id" fail "amy send failed"; return
}
if wait_for_message B "$gid" "joined from amethyst" 30 \
&& wait_for_message C "$gid" "joined from amethyst" 30; then
if wait_for_message B "$gid" "joined from amethyst" 90 \
&& wait_for_message C "$gid" "joined from amethyst" 90; then
record_result "$id" pass
else
record_result "$id" fail "B or C didn't see A's message"
tools/marmot-interop/headless/tests-extras.sh
+36 -28
View File
@@ -7,35 +7,37 @@ test_09_reply_react_unreact() {
banner "Test 09 — reply / react / unreact"
local id="09 reply/react"
local gid; gid=$(load_state GROUP_02 || true)
local gid mls_gid
gid=$(load_state GROUP_02 || true)
mls_gid=$(load_state GROUP_02_MLS || true)
if [[ -z "${gid:-}" ]]; then
record_result "$id" skip "no GROUP_02"; return
fi
# B anchors. Needs a member to be present — if Test 11 already ran and A left,
# skip cleanly so we don't double-fail.
if ! wn_b --json groups members "$gid" 2>/dev/null \
| jq -e --arg p "$A_HEX" '.[]? | select((.pubkey // .public_key) == $p)' \
if ! wn_b --json groups members "$mls_gid" 2>/dev/null \
| jq -e --arg p "$A_HEX" '(.result // .) | .[]? | select((.pubkey // .public_key) == $p)' \
>/dev/null 2>&1; then
record_result "$id" skip "A already left GROUP_02"; return
fi
wn_b messages send "$gid" "anchor for reactions" >/dev/null 2>&1 || true
wn_b messages send "$mls_gid" "anchor for reactions" >/dev/null 2>&1 || true
sleep 3
local msg_id
msg_id=$(wn_b --json messages list "$gid" --limit 10 2>/dev/null \
| jq -r '[.[]? | select((.content // .text // "") == "anchor for reactions")][0].id // empty')
msg_id=$(wn_b --json messages list "$mls_gid" --limit 10 2>/dev/null \
| jq -r '[(.result // .) | .[]? | select((.content // .text // "") == "anchor for reactions")][0].id // empty')
if [[ -z "$msg_id" || "$msg_id" == "null" ]]; then
record_result "$id" fail "couldn't find anchor message id"; return
fi
wn_b messages react "$gid" "$msg_id" "🌮" >/dev/null 2>&1 || true
wn_b messages react "$mls_gid" "$msg_id" "🌮" >/dev/null 2>&1 || true
sleep 3
# amy reply
amy_json marmot message send "$gid" "replying via amy" >/dev/null || {
record_result "$id" fail "amy send reply failed"; return
}
if wait_for_message B "$gid" "replying via amy" 30; then
if wait_for_message B "$mls_gid" "replying via amy" 90; then
record_result "$id" pass
else
record_result "$id" fail "B didn't receive reply"
@@ -48,12 +50,14 @@ test_10_concurrent_commits() {
banner "Test 10 — Concurrent commits race"
local id="10 concurrent commits"
local gid; gid=$(load_state GROUP_02 || true)
local gid mls_gid
gid=$(load_state GROUP_02 || true)
mls_gid=$(load_state GROUP_02_MLS || true)
if [[ -z "${gid:-}" ]]; then
record_result "$id" skip "no GROUP_02"; return
fi
if ! wn_b --json groups members "$gid" 2>/dev/null \
| jq -e --arg p "$A_HEX" '.[]? | select((.pubkey // .public_key) == $p)' \
if ! wn_b --json groups members "$mls_gid" 2>/dev/null \
| jq -e --arg p "$A_HEX" '(.result // .) | .[]? | select((.pubkey // .public_key) == $p)' \
>/dev/null 2>&1; then
record_result "$id" skip "A already left GROUP_02"; return
fi
@@ -62,21 +66,21 @@ test_10_concurrent_commits() {
# guarantees a deterministic outcome.
( amy_json marmot group rename "$gid" "race-from-amethyst" >/dev/null ) &
local a_pid=$!
( wn_b groups rename "$gid" "race-from-wn" >/dev/null 2>&1 ) &
( wn_b groups rename "$mls_gid" "race-from-wn" >/dev/null 2>&1 ) &
local b_pid=$!
wait "$a_pid" "$b_pid" 2>/dev/null || true
sleep 10
local b_name
b_name=$(wn_b --json groups show "$gid" 2>/dev/null | jq -r '.name // empty')
b_name=$(wn_b --json groups show "$mls_gid" 2>/dev/null | jq -r '(.result // .) | .name // empty')
local a_name
a_name=$(amy_field '.name' marmot group show "$gid" 2>/dev/null || echo "")
if [[ -n "$a_name" && "$a_name" == "$b_name" ]]; then
info "race converged: both sides see \"$a_name\""
# Verify encryption still works.
wn_b messages send "$gid" "post-race ping" >/dev/null 2>&1 || true
if amy_json marmot await message "$gid" --match "post-race ping" --timeout 15 >/dev/null; then
wn_b messages send "$mls_gid" "post-race ping" >/dev/null 2>&1 || true
if amy_json marmot await message "$gid" --match "post-race ping" --timeout 90 >/dev/null; then
record_result "$id" pass
else
record_result "$id" fail "encryption broken after race"
@@ -90,35 +94,39 @@ test_12_offline_catchup() {
banner "Test 12 — Offline catch-up"
local id="12 offline catchup"
# Fresh group so we don't collide with other tests.
local out gid
# wn-side keys groups by mls_group_id; amy-side by nostr_group_id. Learn
# the amy side from `amy await group` so later amy calls target the right
# group.
local out mls_gid a_out a_gid
out=$(wn_b --json groups create "Interop-12" "$A_NPUB" 2>>"$LOG_FILE")
gid=$(printf '%s' "$out" | jq_group_id)
[[ -n "$gid" ]] || { record_result "$id" fail "couldn't create Interop-12"; return; }
save_state GROUP_12 "$gid"
mls_gid=$(printf '%s' "$out" | jq_group_id)
[[ -n "$mls_gid" ]] || { record_result "$id" fail "couldn't create Interop-12"; return; }
save_state GROUP_12_MLS "$mls_gid"
# A joins.
amy_json marmot await group --name "Interop-12" --timeout 30 >/dev/null || {
a_out=$(amy_json marmot await group --name "Interop-12" --timeout 30) || {
record_result "$id" fail "A never received Interop-12 invite"; return
}
a_gid=$(printf '%s' "$a_out" | jq -r '.group_id')
save_state GROUP_12 "$a_gid"
# "Go offline" == don't invoke amy. Meanwhile B sends 5 messages + adds C + sends 3 more + rename.
for i in 1 2 3 4 5; do
wn_b messages send "$gid" "offline-msg-$i" >/dev/null 2>&1 || true
wn_b messages send "$mls_gid" "offline-msg-$i" >/dev/null 2>&1 || true
sleep 1
done
wn_b groups add-members "$gid" "$C_NPUB" >/dev/null 2>&1 || true
wait_for_invite C 30 >/dev/null && wn_c groups accept "$gid" >/dev/null 2>&1 || true
wn_b groups add-members "$mls_gid" "$C_NPUB" >/dev/null 2>&1 || true
wait_for_invite C 30 >/dev/null && wn_c groups accept "$mls_gid" >/dev/null 2>&1 || true
for i in 6 7 8; do
wn_b messages send "$gid" "offline-msg-$i" >/dev/null 2>&1 || true
wn_b messages send "$mls_gid" "offline-msg-$i" >/dev/null 2>&1 || true
sleep 1
done
wn_b groups rename "$gid" "Interop-12-renamed" >/dev/null 2>&1 || true
wn_b groups rename "$mls_gid" "Interop-12-renamed" >/dev/null 2>&1 || true
sleep 3
# A comes back online — single sync pulls everything.
local show
show=$(amy_json marmot group show "$gid") || {
show=$(amy_json marmot group show "$a_gid") || {
record_result "$id" fail "amy group show failed"; return
}
local name; name=$(printf '%s' "$show" | jq -r '.name')
@@ -127,7 +135,7 @@ test_12_offline_catchup() {
fi
# All 8 messages should be locally stored.
local msgs; msgs=$(amy_json marmot message list "$gid" --limit 100)
local msgs; msgs=$(amy_json marmot message list "$a_gid" --limit 100)
local missing=0
for i in 1 2 3 4 5 6 7 8; do
if ! printf '%s' "$msgs" | jq -e --arg t "offline-msg-$i" \
tools/marmot-interop/headless/tests-manage.sh
+49 -28
View File
@@ -7,9 +7,17 @@ test_06_member_removal() {
banner "Test 06 — Member removal + forward secrecy"
local id="06 member removal"
local gid; gid=$(load_state GROUP_05 || true)
if [[ -z "${gid:-}" ]]; then
record_result "$id" skip "no GROUP_05"; return
# MIP-03 only admins may commit Remove proposals. In GROUP_05 (wn-created
# by B, A joined later) A is not an admin, so the test used to fail with
# `IllegalStateException: non-admin members may only commit...`. Test on
# GROUP_02 instead, where amy is the creator and therefore sole admin,
# and where test 04 has already added C. amy calls use the nostr id,
# wn calls use the MLS id.
local gid mls_gid
gid=$(load_state GROUP_02 || true)
mls_gid=$(load_state GROUP_02_MLS || true)
if [[ -z "${gid:-}" || -z "${mls_gid:-}" ]]; then
record_result "$id" skip "no GROUP_02"; return
fi
amy_json marmot group remove "$gid" "$C_NPUB" >/dev/null || {
@@ -17,10 +25,10 @@ test_06_member_removal() {
}
# C should no longer see the group on its own member view.
local deadline=$(( $(date +%s) + 60 )) removed=0
local deadline=$(( $(date +%s) + 120 )) removed=0
while [[ $(date +%s) -lt $deadline ]]; do
if ! wn_c --json groups members "$gid" 2>/dev/null \
| jq -e --arg p "$C_HEX" '.[]? | select((.pubkey // .public_key) == $p)' \
if ! wn_c --json groups members "$mls_gid" 2>/dev/null \
| jq -e --arg p "$C_HEX" '(.result // .) | .[]? | select((.pubkey // .public_key) == $p)' \
>/dev/null 2>&1; then
removed=1; break
fi
@@ -33,13 +41,13 @@ test_06_member_removal() {
amy_json marmot message send "$gid" "after removing C" >/dev/null || {
record_result "$id" fail "amy send failed"; return
}
wait_for_message B "$gid" "after removing C" 30 || {
wait_for_message B "$mls_gid" "after removing C" 90 || {
record_result "$id" fail "B lost access after C's removal"; return
}
# Forward secrecy: C must NOT see the post-removal message.
sleep 5
if wait_for_message C "$gid" "after removing C" 10; then
if wait_for_message C "$mls_gid" "after removing C" 10; then
record_result "$id" fail "C still decrypted a post-removal message"
else
record_result "$id" pass
@@ -50,8 +58,13 @@ test_07_metadata_rename() {
banner "Test 07 — Metadata rename round-trip (MIP-01)"
local id="07 metadata rename"
local gid; gid=$(load_state GROUP_02 || true)
if [[ -z "${gid:-}" ]]; then
# amy was the creator of GROUP_02 so its own nostr_group_id is saved as
# GROUP_02; wn keys its copy by the MLS group id saved as GROUP_02_MLS.
# Pass each CLI the id it understands.
local gid mls_gid
gid=$(load_state GROUP_02 || true)
mls_gid=$(load_state GROUP_02_MLS || true)
if [[ -z "${gid:-}" || -z "${mls_gid:-}" ]]; then
record_result "$id" skip "no GROUP_02"; return
fi
@@ -59,9 +72,9 @@ test_07_metadata_rename() {
record_result "$id" fail "amy rename failed"; return
}
local deadline=$(( $(date +%s) + 60 )) seen=""
local deadline=$(( $(date +%s) + 120 )) seen=""
while [[ $(date +%s) -lt $deadline ]]; do
seen=$(wn_b --json groups show "$gid" 2>/dev/null | jq -r '.name // empty')
seen=$(wn_b --json groups show "$mls_gid" 2>/dev/null | jq -r '(.result // .) | .name // empty')
[[ "$seen" == "Interop-02-renamed" ]] && break
sleep 3
done
@@ -70,8 +83,8 @@ test_07_metadata_rename() {
}
# Now B renames back and A should pick it up.
wn_b groups rename "$gid" "Interop-02-reverse" >/dev/null 2>&1 || true
if amy_json marmot await rename "$gid" --name "Interop-02-reverse" --timeout 60 >/dev/null; then
wn_b groups rename "$mls_gid" "Interop-02-reverse" >/dev/null 2>&1 || true
if amy_json marmot await rename "$gid" --name "Interop-02-reverse" --timeout 120 >/dev/null; then
record_result "$id" pass
else
record_result "$id" fail "A did not pick up B's rename"
@@ -82,39 +95,45 @@ test_08_admin_promote_demote() {
banner "Test 08 — Admin promote / demote"
local id="08 admin promote/demote"
local gid; gid=$(load_state GROUP_03 || true)
if [[ -z "${gid:-}" ]]; then
# GROUP_03 was created by wn so both sides need different ids:
# GROUP_03 → amy's nostr_group_id (captured in test 03 after
# `amy await group` returned `.group_id`)
# GROUP_03_MLS → wn's mls_group_id (wn's `groups create` output)
local a_gid mls_gid
a_gid=$(load_state GROUP_03 || true)
mls_gid=$(load_state GROUP_03_MLS || true)
if [[ -z "${mls_gid:-}" ]]; then
record_result "$id" skip "no GROUP_03"; return
fi
# Ensure 3 members (add C if missing).
wn_c keys publish >/dev/null 2>&1 || true
sleep 2
wn_b groups add-members "$gid" "$C_NPUB" >/dev/null 2>&1 || true
wait_for_invite C 30 >/dev/null && wn_c groups accept "$gid" >/dev/null 2>&1 || true
wn_b groups add-members "$mls_gid" "$C_NPUB" >/dev/null 2>&1 || true
wait_for_invite C 30 >/dev/null && wn_c groups accept "$mls_gid" >/dev/null 2>&1 || true
# B promotes A.
wn_b groups promote "$gid" "$A_NPUB" >/dev/null 2>&1 || {
wn_b groups promote "$mls_gid" "$A_NPUB" >/dev/null 2>&1 || {
record_result "$id" fail "wn promote failed"; return
}
# A should reflect the new admin set — poll via amy.
if ! amy_json marmot await admin "$gid" "$A_NPUB" --timeout 30 >/dev/null; then
if ! amy_json marmot await admin "$a_gid" "$A_NPUB" --timeout 90 >/dev/null; then
record_result "$id" fail "A never saw itself promoted"; return
fi
# A now commits a rename — only possible if we're admin.
amy_json marmot group rename "$gid" "Interop-03-by-A" >/dev/null || {
amy_json marmot group rename "$a_gid" "Interop-03-by-A" >/dev/null || {
record_result "$id" fail "A (now admin) could not rename"; return
}
# B demotes A.
wn_b groups demote "$gid" "$A_NPUB" >/dev/null 2>&1 || warn "demote returned nonzero"
wn_b groups demote "$mls_gid" "$A_NPUB" >/dev/null 2>&1 || warn "demote returned nonzero"
sleep 5
local admins
admins=$(wn_b --json groups admins "$gid" 2>/dev/null \
| jq -r '.[].pubkey // .[].public_key // .[]' | tr '\n' ' ')
admins=$(wn_b --json groups admins "$mls_gid" 2>/dev/null \
| jq -r '(.result // .) | .[]?.pubkey // .[]?.public_key // .[]?' | tr '\n' ' ')
if [[ "$admins" == *"$A_HEX"* ]]; then
record_result "$id" fail "A still admin after demote"
else
@@ -126,7 +145,9 @@ test_11_leave_group() {
banner "Test 11 — Leave group"
local id="11 leave group"
local gid; gid=$(load_state GROUP_02 || true)
local gid mls_gid
gid=$(load_state GROUP_02 || true)
mls_gid=$(load_state GROUP_02_MLS || true)
if [[ -z "${gid:-}" ]]; then
record_result "$id" skip "no GROUP_02"; return
fi
@@ -135,10 +156,10 @@ test_11_leave_group() {
record_result "$id" fail "amy leave failed"; return
}
local deadline=$(( $(date +%s) + 60 )) gone=0
local deadline=$(( $(date +%s) + 120 )) gone=0
while [[ $(date +%s) -lt $deadline ]]; do
if ! wn_b --json groups members "$gid" 2>/dev/null \
| jq -e --arg p "$A_HEX" '.[]? | select((.pubkey // .public_key) == $p)' \
if ! wn_b --json groups members "$mls_gid" 2>/dev/null \
| jq -e --arg p "$A_HEX" '(.result // .) | .[]? | select((.pubkey // .public_key) == $p)' \
>/dev/null 2>&1; then
gone=1; break
fi
tools/marmot-interop/lib.sh
+13 -5
View File
@@ -150,6 +150,10 @@ expect_contains() {
# - plain hex string (from `groups list`)
# - {"value":{"vec":[...]}} serde struct (from `groups create`)
# - flat byte array [n, ...] (from some responses)
# Plus the three wrapper shapes wn actually uses:
# - {"result": {"mls_group_id": ...}} (groups create)
# - {"group": {"mls_group_id": ...}, "membership": ...} (groups invites[0])
# - {"mls_group_id": ...} (bare)
# Input: JSON string via stdin; optional 2nd arg = field name (default: mls_group_id)
jq_group_id() {
local field="${1:-mls_group_id}"
@@ -159,7 +163,8 @@ jq_group_id() {
[($n / 16 | floor), ($n % 16)] |
map(if . < 10 then (48 + .) else (87 + .) end) |
implode;
(.result // .) |
(.group // .result // .) |
(.group // .) |
.[$f] |
if type == "string" then .
elif (type == "object" and (.value.vec != null)) then
@@ -190,8 +195,11 @@ wait_for_invite() {
deadline=$(( start + timeout ))
last_hb=$start
while [[ $(date +%s) -lt $deadline ]]; do
# Post-v0.2 `wn --json groups invites` returns `{"result": [...]}`
# (older builds returned the bare array). Peel the wrapper when
# present so a pending invite is actually detected.
gid=$("$wnfn" --json groups invites 2>/dev/null \
| jq -c '.[0] // empty' 2>/dev/null | jq_group_id || true)
| jq -c '(.result // .) | .[0] // empty' 2>/dev/null | jq_group_id || true)
if [[ -n "${gid:-}" ]]; then
printf '%s\n' "$gid"
return 0
@@ -203,7 +211,7 @@ wait_for_invite() {
local elapsed=$(( now - start )) remaining=$(( deadline - now ))
local pending
pending=$("$wnfn" --json groups invites 2>/dev/null \
| jq 'length' 2>/dev/null || echo "?")
| jq '(.result // .) | length' 2>/dev/null || echo "?")
local recent=""
if [[ -f "$data_dir/logs/stderr.log" ]]; then
recent=$(tail -n 200 "$data_dir/logs/stderr.log" 2>/dev/null \
@@ -233,7 +241,7 @@ wait_for_message() {
fi
if [[ -n "${payload:-}" ]] && \
printf '%s' "$payload" | jq -e --arg n "$needle" \
'.[]? | select((.content // .text // "") | contains($n))' \
'(.result // .) | .[]? | select((.content // .text // "") | contains($n))' \
>/dev/null 2>&1; then
return 0
fi
@@ -254,7 +262,7 @@ wait_for_member() {
payload=$(wn_c_json groups members "$gid" 2>/dev/null || true)
fi
if printf '%s' "${payload:-}" | jq -e --arg p "$pubkey" \
'.[]? | select((.pubkey // .public_key // "") == $p)' \
'(.result // .) | .[]? | select((.pubkey // .public_key // "") == $p)' \
>/dev/null 2>&1; then
return 0
fi