Files
amethyst/tools/marmot-interop/headless/tests-create.sh
T
Claude f08b010f50 fix(marmot,cli,interop): interop-compatible Marmot flows and harness correctness
Five protocol-level fixes and a batch of harness correctness fixes to get
the headless Marmot/Whitenoise interop harness from 1/13 to 5/13 passing
cleanly, with the remaining failures all rooted in wn's per-account
serial event-processor retry backoff (which drops undecryptable
pre-membership commits after several minutes) rather than amy behaviour.

quartz + commons
----------------
* MarmotGroupData: hold CURRENT_VERSION at 2. mdk-core (the Rust MLS
  engine used by whitenoise-rs) strict-rejects v3 payloads with
  `ExtensionFormatError("Trailing bytes in NostrGroupDataExtension")`
  — our v3 welcomes and GCE commits never apply, so every cross-client
  group flow dies at welcome processing. We still parse v3 happily on
  the way in; we just don't emit it until mdk publishes the
  forward-compat fix MIP-01 mandates.
* MarmotManager.updateGroupMetadata: MERGE extensions instead of
  REPLACING. RFC 9420 §12.1.7 says GCE proposals blow away the old
  extension list; callers that pass only [marmot_group_data] dropped
  [required_capabilities], which peers then reject. Preserve every slot
  except the one we're updating.
* MarmotManager.createGroup: new optional `initialMetadata` parameter
  that bakes MarmotGroupData into epoch-0 GroupContext.extensions
  directly. Without it, creators had to publish a pre-membership
  "bootstrap" commit that no later joiner could decrypt — each such
  peer then burned their retry budget on an undecryptable kind:445
  before seeing the real state. Threaded through MlsGroup.create /
  MlsGroupManager.createGroup.
* MarmotManager.mlsGroupIdHex: new translation helper so any code
  juggling the MIP-01 nostr_group_id (what amy indexes on) and the
  MLS GroupContext groupId (what mdk indexes on) can cross-reference
  them without reaching into MlsGroupManager directly.

amethyst module
---------------
* Account.leaveMarmotGroup: self-demote before SelfRemove per MIP-01,
  and promote a surviving member to admin first if the caller is the
  sole admin (otherwise we'd throw "admin depletion"). Matches the
  cli/GroupMembershipCommands.leave flow.

cli (amy)
---------
* Context.syncIncoming: don't advance `giftWrapSince` on empty polls
  (so the first-ever sync doesn't bump the cursor past every
  past-timestamped wrap we've ever been sent), subtract 2 days lookback
  when filtering (NIP-59 randomWithTwoDays gift wraps can have any
  createdAt in the last 48h), and only advance `groupSince` for groups
  we actually received events for.
* Context.syncIncoming: after ingest, if any Welcome consumed a
  KeyPackage, rotate and publish a fresh one immediately. MIP-00
  requires this — a KP can only be welcomed once and leaving the
  consumed one on relays just means later senders invite us with a
  bundle we no longer have private keys for.
* Context.resolveGroupId: accept either nostr_group_id (amy's primary
  key) or the MLS GroupContext groupId (what wn emits) on every verb
  that takes a group id. Wired through GroupAdd/Remove/Leave/Metadata
  /Read, Message send/list, and all the await* verbs so harness
  scripts never have to juggle both forms for a single group.
* GroupCreateCommand: bake initial metadata into epoch 0 (see the
  quartz change above). Dropped the now-redundant bootstrap commit
  publish and tightened the JSON output to include `mls_group_id`.
* GroupMembershipCommands.leave: self-demote admin before SelfRemove;
  promote an heir if we're the only admin, otherwise the GCE would
  deplete admins and the leave aborts.
* MarmotIngest.ingestGiftWrap: unwrap the sealed-rumor layer. NIP-59
  wrap is gift-wrap(kind:1059) → seal(kind:13) → rumor; the old code
  only unwrapped once and then checked `inner.kind == 444`, which is
  always false because inner is actually the seal. Unseal once more
  before the Welcome check. This single fix is what unsticks every
  amy-side Welcome ingestion.

wn harness patches + scripts
----------------------------
* whitenoise-defaults-env.patch: honour $WHITENOISE_DISCOVERY_RELAYS in
  `Relay::defaults()` (release builds otherwise bake damus.io / primal
  / nos.lol into every new account's NIP-65 / Inbox / KeyPackage
  lists, which breaks publishing and prevents the inbox subscription
  plane from ever reaching an operational state in a sandbox).
* setup.sh: sleep 2s after amy's initial kind:30443 publish so
  nostr-rs-relay has a chance to fsync before wn's first targeted
  discovery query. Without it wn's `keys check` races the relay's
  WAL flush and intermittently returns NotFound.
* lib.sh: peel wn's `{"result": …}` wrapper in `jq_group_id`,
  `wait_for_invite`, `wait_for_message`, `wait_for_member`. Post-v0.2
  wn `--json` output nests everything under `.result` (and
  `groups invites[]` nests further under `.group.mls_group_id`) —
  these helpers were still pattern-matching on the flat shape, so
  they returned empty strings for a perfectly good response.
* tests-{create,manage,extras}.sh: track both group IDs per test
  (amy's nostr + wn's MLS), pass each CLI the id it understands, and
  bump the post-commit wait timeouts to 90–120s so wn's exponential
  retry backoff has time to work through the pre-membership commits
  it can't decrypt and get to the ones it can.

https://claude.ai/code/session_016kAxdp6ubB5CnF9URhCEzP
2026-04-22 00:28:45 +00:00

200 lines
7.1 KiB
Bash

# shellcheck shell=bash
#
# headless/tests-create.sh — tests 01..05.
# Focus: KeyPackage discovery, group creation, invites, initial messages.
test_01_keypackage_discovery() {
banner "Test 01 — KeyPackage publish & discovery (MIP-00)"
local id="01 KeyPackage A<->B"
# B finds A's KP
local raw ev
raw=$(wn_b --json keys check "$A_NPUB" 2>>"$LOG_FILE" || true)
ev=$(printf '%s' "$raw" | jq -r '.result.event_id // .event_id // empty')
if [[ -z "$ev" || "$ev" == "null" ]]; then
record_result "$id (B->A)" fail "wn couldn't find A's KP"; return
fi
info "B saw A's KP $ev"
# A finds B's KP (wn publishes automatically via create-identity flow? if not, ask).
wn_b keys publish >>"$LOG_FILE" 2>&1 || warn "wn_b keys publish failed"
sleep 3
local out
out=$(amy_json marmot key-package check "$B_NPUB" 2>/dev/null || true)
if [[ -z "$out" ]]; then
record_result "$id (A->B)" fail "amy couldn't find B's KP"; return
fi
info "A saw B's KP $(printf '%s' "$out" | jq -r .event_id)"
record_result "$id" pass
}
test_02_a_creates_group() {
banner "Test 02 — A creates group, invites B"
local id="02 A->B create+invite"
# amy keys groups by the MIP-01 nostr_group_id (`group_id`), wn (via
# mdk-core) keys by the MLS GroupContext groupId (`mls_group_id`). Both
# are 32 random bytes and they are NOT the same. We need both: amy calls
# use `gid`, wn calls use `mls_gid`.
local out gid mls_gid
out=$(amy_json marmot group create --name "Interop-02") || {
record_result "$id" fail "create returned no group_id"; return
}
gid=$(printf '%s' "$out" | jq -r '.group_id')
mls_gid=$(printf '%s' "$out" | jq -r '.mls_group_id')
save_state GROUP_02 "$gid"
save_state GROUP_02_MLS "$mls_gid"
info "created group nostr=$gid mls=$mls_gid"
amy_json marmot group add "$gid" "$B_NPUB" >/dev/null || {
record_result "$id" fail "amy group add failed"; return
}
# B waits for invite, accepts.
local b_gid
if ! b_gid=$(wait_for_invite B 60); then
record_result "$id" fail "B never received invite"; return
fi
wn_b groups accept "$b_gid" >/dev/null 2>&1 || true
info "B joined $b_gid"
# A -> B message. amy takes the nostr id; wait_for_message calls
# `wn messages list` which needs the MLS id.
amy_json marmot message send "$gid" "hello from amethyst" >/dev/null || {
record_result "$id" fail "amy send failed"; return
}
if ! wait_for_message B "$mls_gid" "hello from amethyst" 90; then
record_result "$id" fail "B didn't receive A's message"; return
fi
# B -> A message
wn_b messages send "$mls_gid" "hello from wn" >/dev/null 2>&1 || warn "wn send returned nonzero"
if ! amy_json marmot await message "$gid" --match "hello from wn" --timeout 30 >/dev/null; then
record_result "$id" fail "A didn't receive B's reply"; return
fi
record_result "$id" pass
}
test_03_b_creates_group() {
banner "Test 03 — B creates group, invites A"
local id="03 B->A create+invite"
# `wn groups create` returns the MLS group id (what wn's messages API
# expects). amy indexes by the MIP-01 nostr_group_id, which we only learn
# once A has processed the welcome and we can ask `amy await group` for
# it. Keep them distinct so each CLI gets the id it understands.
local out mls_gid
out=$(wn_b --json groups create "Interop-03" "$A_NPUB" 2>>"$LOG_FILE") || {
record_result "$id" fail "wn groups create failed"; return
}
mls_gid=$(printf '%s' "$out" | jq_group_id)
if [[ -z "$mls_gid" ]]; then
record_result "$id" fail "could not parse group_id"; return
fi
save_state GROUP_03_MLS "$mls_gid"
info "mls_group_id: $mls_gid"
# A: poll until it joins, and capture its nostr_group_id.
local a_out a_gid
a_out=$(amy_json marmot await group --name "Interop-03" --timeout 30) || {
record_result "$id" fail "A never joined B's group"; return
}
a_gid=$(printf '%s' "$a_out" | jq -r '.group_id')
save_state GROUP_03 "$a_gid"
info "A joined as nostr_group_id=$a_gid"
# B -> A message
wn_b messages send "$mls_gid" "ping from wn" >/dev/null 2>&1 || true
if ! amy_json marmot await message "$a_gid" --match "ping from wn" --timeout 30 >/dev/null; then
record_result "$id" fail "A didn't see B's ping"; return
fi
# A -> B reply
amy_json marmot message send "$a_gid" "pong from amethyst" >/dev/null || {
record_result "$id" fail "amy send pong failed"; return
}
if wait_for_message B "$mls_gid" "pong from amethyst" 90; then
record_result "$id" pass
else
record_result "$id" fail "B didn't see A's pong"
fi
}
test_04_three_member_group() {
banner "Test 04 — 3-member group, add C after create"
local id="04 3-member add-after-create"
wn_c keys publish >/dev/null 2>&1 || true
sleep 3
local gid mls_gid
gid=$(load_state GROUP_02 || true)
mls_gid=$(load_state GROUP_02_MLS || true)
if [[ -z "${gid:-}" ]]; then
record_result "$id" skip "no GROUP_02"; return
fi
# A adds C
amy_json marmot group add "$gid" "$C_NPUB" >/dev/null || {
record_result "$id" fail "amy group add C failed"; return
}
local c_gid
if ! c_gid=$(wait_for_invite C 60); then
record_result "$id" fail "C never received invite"; return
fi
wn_c groups accept "$c_gid" >/dev/null 2>&1 || true
amy_json marmot message send "$gid" "hello three-member world" >/dev/null || {
record_result "$id" fail "amy send failed"; return
}
# wn's event_processor retries undecryptable pre-membership commits with
# exponential backoff (2+4+8+16=~30s), and kind:445 processing is serial
# per-account; a fresh joiner routinely needs ~60s to burn through that
# retry queue before the add-C commit is applied and "hello three-member
# world" decrypts. The 30s we used for the inline A<->B send/recv
# wouldn't clear that.
if wait_for_message B "$mls_gid" "hello three-member world" 90 \
&& wait_for_message C "$c_gid" "hello three-member world" 90; then
record_result "$id" pass
else
record_result "$id" fail "B or C missed A's post-add message"
fi
}
test_05_b_adds_a_existing() {
banner "Test 05 — B adds A to an existing B+C group"
local id="05 wn adds A existing"
local out gid
out=$(wn_b --json groups create "Interop-05" "$C_NPUB" 2>>"$LOG_FILE") || {
record_result "$id" fail "wn create Interop-05 failed"; return
}
gid=$(printf '%s' "$out" | jq_group_id)
if [[ -z "$gid" ]]; then
record_result "$id" fail "no group_id from create"; return
fi
save_state GROUP_05 "$gid"
wait_for_invite C 30 >/dev/null && wn_c groups accept "$gid" >/dev/null 2>&1 || true
wn_b groups add-members "$gid" "$A_NPUB" >/dev/null 2>&1 || {
record_result "$id" fail "wn add-members A failed"; return
}
# A joins
if ! amy_json marmot await group --name "Interop-05" --timeout 30 >/dev/null; then
record_result "$id" fail "A never received invite to Interop-05"; return
fi
amy_json marmot message send "$gid" "joined from amethyst" >/dev/null || {
record_result "$id" fail "amy send failed"; return
}
if wait_for_message B "$gid" "joined from amethyst" 90 \
&& wait_for_message C "$gid" "joined from amethyst" 90; then
record_result "$id" pass
else
record_result "$id" fail "B or C didn't see A's message"
fi
}