fix(marmot,cli,interop): interop-compatible Marmot flows and harness correctness
Five protocol-level fixes and a batch of harness correctness fixes to get
the headless Marmot/Whitenoise interop harness from 1/13 to 5/13 passing
cleanly, with the remaining failures all rooted in wn's per-account
serial event-processor retry backoff (which drops undecryptable
pre-membership commits after several minutes) rather than amy behaviour.
quartz + commons
----------------
* MarmotGroupData: hold CURRENT_VERSION at 2. mdk-core (the Rust MLS
engine used by whitenoise-rs) strict-rejects v3 payloads with
`ExtensionFormatError("Trailing bytes in NostrGroupDataExtension")`
— our v3 welcomes and GCE commits never apply, so every cross-client
group flow dies at welcome processing. We still parse v3 happily on
the way in; we just don't emit it until mdk publishes the
forward-compat fix MIP-01 mandates.
* MarmotManager.updateGroupMetadata: MERGE extensions instead of
REPLACING. RFC 9420 §12.1.7 says GCE proposals blow away the old
extension list; callers that pass only [marmot_group_data] dropped
[required_capabilities], which peers then reject. Preserve every slot
except the one we're updating.
* MarmotManager.createGroup: new optional `initialMetadata` parameter
that bakes MarmotGroupData into epoch-0 GroupContext.extensions
directly. Without it, creators had to publish a pre-membership
"bootstrap" commit that no later joiner could decrypt — each such
peer then burned their retry budget on an undecryptable kind:445
before seeing the real state. Threaded through MlsGroup.create /
MlsGroupManager.createGroup.
* MarmotManager.mlsGroupIdHex: new translation helper so any code
juggling the MIP-01 nostr_group_id (what amy indexes on) and the
MLS GroupContext groupId (what mdk indexes on) can cross-reference
them without reaching into MlsGroupManager directly.
amethyst module
---------------
* Account.leaveMarmotGroup: self-demote before SelfRemove per MIP-01,
and promote a surviving member to admin first if the caller is the
sole admin (otherwise we'd throw "admin depletion"). Matches the
cli/GroupMembershipCommands.leave flow.
cli (amy)
---------
* Context.syncIncoming: don't advance `giftWrapSince` on empty polls
(so the first-ever sync doesn't bump the cursor past every
past-timestamped wrap we've ever been sent), subtract 2 days lookback
when filtering (NIP-59 randomWithTwoDays gift wraps can have any
createdAt in the last 48h), and only advance `groupSince` for groups
we actually received events for.
* Context.syncIncoming: after ingest, if any Welcome consumed a
KeyPackage, rotate and publish a fresh one immediately. MIP-00
requires this — a KP can only be welcomed once and leaving the
consumed one on relays just means later senders invite us with a
bundle we no longer have private keys for.
* Context.resolveGroupId: accept either nostr_group_id (amy's primary
key) or the MLS GroupContext groupId (what wn emits) on every verb
that takes a group id. Wired through GroupAdd/Remove/Leave/Metadata
/Read, Message send/list, and all the await* verbs so harness
scripts never have to juggle both forms for a single group.
* GroupCreateCommand: bake initial metadata into epoch 0 (see the
quartz change above). Dropped the now-redundant bootstrap commit
publish and tightened the JSON output to include `mls_group_id`.
* GroupMembershipCommands.leave: self-demote admin before SelfRemove;
promote an heir if we're the only admin, otherwise the GCE would
deplete admins and the leave aborts.
* MarmotIngest.ingestGiftWrap: unwrap the sealed-rumor layer. NIP-59
wrap is gift-wrap(kind:1059) → seal(kind:13) → rumor; the old code
only unwrapped once and then checked `inner.kind == 444`, which is
always false because inner is actually the seal. Unseal once more
before the Welcome check. This single fix is what unsticks every
amy-side Welcome ingestion.
wn harness patches + scripts
----------------------------
* whitenoise-defaults-env.patch: honour $WHITENOISE_DISCOVERY_RELAYS in
`Relay::defaults()` (release builds otherwise bake damus.io / primal
/ nos.lol into every new account's NIP-65 / Inbox / KeyPackage
lists, which breaks publishing and prevents the inbox subscription
plane from ever reaching an operational state in a sandbox).
* setup.sh: sleep 2s after amy's initial kind:30443 publish so
nostr-rs-relay has a chance to fsync before wn's first targeted
discovery query. Without it wn's `keys check` races the relay's
WAL flush and intermittently returns NotFound.
* lib.sh: peel wn's `{"result": …}` wrapper in `jq_group_id`,
`wait_for_invite`, `wait_for_message`, `wait_for_member`. Post-v0.2
wn `--json` output nests everything under `.result` (and
`groups invites[]` nests further under `.group.mls_group_id`) —
these helpers were still pattern-matching on the flat shape, so
they returned empty strings for a perfectly good response.
* tests-{create,manage,extras}.sh: track both group IDs per test
(amy's nostr + wn's MLS), pass each CLI the id it understands, and
bump the post-commit wait timeouts to 90–120s so wn's exponential
retry backoff has time to work through the pre-membership commits
it can't decrypt and get to the ones it can.
https://claude.ai/code/session_016kAxdp6ubB5CnF9URhCEzP
This commit is contained in:
Claude
@@ -7,35 +7,37 @@ test_09_reply_react_unreact() {
|
||||
banner "Test 09 — reply / react / unreact"
|
||||
local id="09 reply/react"
|
||||
|
||||
local gid; gid=$(load_state GROUP_02 || true)
|
||||
local gid mls_gid
|
||||
gid=$(load_state GROUP_02 || true)
|
||||
mls_gid=$(load_state GROUP_02_MLS || true)
|
||||
if [[ -z "${gid:-}" ]]; then
|
||||
record_result "$id" skip "no GROUP_02"; return
|
||||
fi
|
||||
|
||||
# B anchors. Needs a member to be present — if Test 11 already ran and A left,
|
||||
# skip cleanly so we don't double-fail.
|
||||
if ! wn_b --json groups members "$gid" 2>/dev/null \
|
||||
| jq -e --arg p "$A_HEX" '.[]? | select((.pubkey // .public_key) == $p)' \
|
||||
if ! wn_b --json groups members "$mls_gid" 2>/dev/null \
|
||||
| jq -e --arg p "$A_HEX" '(.result // .) | .[]? | select((.pubkey // .public_key) == $p)' \
|
||||
>/dev/null 2>&1; then
|
||||
record_result "$id" skip "A already left GROUP_02"; return
|
||||
fi
|
||||
|
||||
wn_b messages send "$gid" "anchor for reactions" >/dev/null 2>&1 || true
|
||||
wn_b messages send "$mls_gid" "anchor for reactions" >/dev/null 2>&1 || true
|
||||
sleep 3
|
||||
local msg_id
|
||||
msg_id=$(wn_b --json messages list "$gid" --limit 10 2>/dev/null \
|
||||
| jq -r '[.[]? | select((.content // .text // "") == "anchor for reactions")][0].id // empty')
|
||||
msg_id=$(wn_b --json messages list "$mls_gid" --limit 10 2>/dev/null \
|
||||
| jq -r '[(.result // .) | .[]? | select((.content // .text // "") == "anchor for reactions")][0].id // empty')
|
||||
if [[ -z "$msg_id" || "$msg_id" == "null" ]]; then
|
||||
record_result "$id" fail "couldn't find anchor message id"; return
|
||||
fi
|
||||
|
||||
wn_b messages react "$gid" "$msg_id" "🌮" >/dev/null 2>&1 || true
|
||||
wn_b messages react "$mls_gid" "$msg_id" "🌮" >/dev/null 2>&1 || true
|
||||
sleep 3
|
||||
# amy reply
|
||||
amy_json marmot message send "$gid" "replying via amy" >/dev/null || {
|
||||
record_result "$id" fail "amy send reply failed"; return
|
||||
}
|
||||
if wait_for_message B "$gid" "replying via amy" 30; then
|
||||
if wait_for_message B "$mls_gid" "replying via amy" 90; then
|
||||
record_result "$id" pass
|
||||
else
|
||||
record_result "$id" fail "B didn't receive reply"
|
||||
@@ -48,12 +50,14 @@ test_10_concurrent_commits() {
|
||||
banner "Test 10 — Concurrent commits race"
|
||||
local id="10 concurrent commits"
|
||||
|
||||
local gid; gid=$(load_state GROUP_02 || true)
|
||||
local gid mls_gid
|
||||
gid=$(load_state GROUP_02 || true)
|
||||
mls_gid=$(load_state GROUP_02_MLS || true)
|
||||
if [[ -z "${gid:-}" ]]; then
|
||||
record_result "$id" skip "no GROUP_02"; return
|
||||
fi
|
||||
if ! wn_b --json groups members "$gid" 2>/dev/null \
|
||||
| jq -e --arg p "$A_HEX" '.[]? | select((.pubkey // .public_key) == $p)' \
|
||||
if ! wn_b --json groups members "$mls_gid" 2>/dev/null \
|
||||
| jq -e --arg p "$A_HEX" '(.result // .) | .[]? | select((.pubkey // .public_key) == $p)' \
|
||||
>/dev/null 2>&1; then
|
||||
record_result "$id" skip "A already left GROUP_02"; return
|
||||
fi
|
||||
@@ -62,21 +66,21 @@ test_10_concurrent_commits() {
|
||||
# guarantees a deterministic outcome.
|
||||
( amy_json marmot group rename "$gid" "race-from-amethyst" >/dev/null ) &
|
||||
local a_pid=$!
|
||||
( wn_b groups rename "$gid" "race-from-wn" >/dev/null 2>&1 ) &
|
||||
( wn_b groups rename "$mls_gid" "race-from-wn" >/dev/null 2>&1 ) &
|
||||
local b_pid=$!
|
||||
wait "$a_pid" "$b_pid" 2>/dev/null || true
|
||||
|
||||
sleep 10
|
||||
local b_name
|
||||
b_name=$(wn_b --json groups show "$gid" 2>/dev/null | jq -r '.name // empty')
|
||||
b_name=$(wn_b --json groups show "$mls_gid" 2>/dev/null | jq -r '(.result // .) | .name // empty')
|
||||
local a_name
|
||||
a_name=$(amy_field '.name' marmot group show "$gid" 2>/dev/null || echo "")
|
||||
|
||||
if [[ -n "$a_name" && "$a_name" == "$b_name" ]]; then
|
||||
info "race converged: both sides see \"$a_name\""
|
||||
# Verify encryption still works.
|
||||
wn_b messages send "$gid" "post-race ping" >/dev/null 2>&1 || true
|
||||
if amy_json marmot await message "$gid" --match "post-race ping" --timeout 15 >/dev/null; then
|
||||
wn_b messages send "$mls_gid" "post-race ping" >/dev/null 2>&1 || true
|
||||
if amy_json marmot await message "$gid" --match "post-race ping" --timeout 90 >/dev/null; then
|
||||
record_result "$id" pass
|
||||
else
|
||||
record_result "$id" fail "encryption broken after race"
|
||||
@@ -90,35 +94,39 @@ test_12_offline_catchup() {
|
||||
banner "Test 12 — Offline catch-up"
|
||||
local id="12 offline catchup"
|
||||
|
||||
# Fresh group so we don't collide with other tests.
|
||||
local out gid
|
||||
# wn-side keys groups by mls_group_id; amy-side by nostr_group_id. Learn
|
||||
# the amy side from `amy await group` so later amy calls target the right
|
||||
# group.
|
||||
local out mls_gid a_out a_gid
|
||||
out=$(wn_b --json groups create "Interop-12" "$A_NPUB" 2>>"$LOG_FILE")
|
||||
gid=$(printf '%s' "$out" | jq_group_id)
|
||||
[[ -n "$gid" ]] || { record_result "$id" fail "couldn't create Interop-12"; return; }
|
||||
save_state GROUP_12 "$gid"
|
||||
mls_gid=$(printf '%s' "$out" | jq_group_id)
|
||||
[[ -n "$mls_gid" ]] || { record_result "$id" fail "couldn't create Interop-12"; return; }
|
||||
save_state GROUP_12_MLS "$mls_gid"
|
||||
|
||||
# A joins.
|
||||
amy_json marmot await group --name "Interop-12" --timeout 30 >/dev/null || {
|
||||
a_out=$(amy_json marmot await group --name "Interop-12" --timeout 30) || {
|
||||
record_result "$id" fail "A never received Interop-12 invite"; return
|
||||
}
|
||||
a_gid=$(printf '%s' "$a_out" | jq -r '.group_id')
|
||||
save_state GROUP_12 "$a_gid"
|
||||
|
||||
# "Go offline" == don't invoke amy. Meanwhile B sends 5 messages + adds C + sends 3 more + rename.
|
||||
for i in 1 2 3 4 5; do
|
||||
wn_b messages send "$gid" "offline-msg-$i" >/dev/null 2>&1 || true
|
||||
wn_b messages send "$mls_gid" "offline-msg-$i" >/dev/null 2>&1 || true
|
||||
sleep 1
|
||||
done
|
||||
wn_b groups add-members "$gid" "$C_NPUB" >/dev/null 2>&1 || true
|
||||
wait_for_invite C 30 >/dev/null && wn_c groups accept "$gid" >/dev/null 2>&1 || true
|
||||
wn_b groups add-members "$mls_gid" "$C_NPUB" >/dev/null 2>&1 || true
|
||||
wait_for_invite C 30 >/dev/null && wn_c groups accept "$mls_gid" >/dev/null 2>&1 || true
|
||||
for i in 6 7 8; do
|
||||
wn_b messages send "$gid" "offline-msg-$i" >/dev/null 2>&1 || true
|
||||
wn_b messages send "$mls_gid" "offline-msg-$i" >/dev/null 2>&1 || true
|
||||
sleep 1
|
||||
done
|
||||
wn_b groups rename "$gid" "Interop-12-renamed" >/dev/null 2>&1 || true
|
||||
wn_b groups rename "$mls_gid" "Interop-12-renamed" >/dev/null 2>&1 || true
|
||||
sleep 3
|
||||
|
||||
# A comes back online — single sync pulls everything.
|
||||
local show
|
||||
show=$(amy_json marmot group show "$gid") || {
|
||||
show=$(amy_json marmot group show "$a_gid") || {
|
||||
record_result "$id" fail "amy group show failed"; return
|
||||
}
|
||||
local name; name=$(printf '%s' "$show" | jq -r '.name')
|
||||
@@ -127,7 +135,7 @@ test_12_offline_catchup() {
|
||||
fi
|
||||
|
||||
# All 8 messages should be locally stored.
|
||||
local msgs; msgs=$(amy_json marmot message list "$gid" --limit 100)
|
||||
local msgs; msgs=$(amy_json marmot message list "$a_gid" --limit 100)
|
||||
local missing=0
|
||||
for i in 1 2 3 4 5 6 7 8; do
|
||||
if ! printf '%s' "$msgs" | jq -e --arg t "offline-msg-$i" \
|
||||
|
||||
Reference in New Issue
Block a user