amethyst

nick/amethyst

Fork 0

Commit Graph

Author	SHA1	Message	Date
Claude	f08b010f50	fix(marmot,cli,interop): interop-compatible Marmot flows and harness correctness Five protocol-level fixes and a batch of harness correctness fixes to get the headless Marmot/Whitenoise interop harness from 1/13 to 5/13 passing cleanly, with the remaining failures all rooted in wn's per-account serial event-processor retry backoff (which drops undecryptable pre-membership commits after several minutes) rather than amy behaviour. quartz + commons ---------------- * MarmotGroupData: hold CURRENT_VERSION at 2. mdk-core (the Rust MLS engine used by whitenoise-rs) strict-rejects v3 payloads with `ExtensionFormatError("Trailing bytes in NostrGroupDataExtension")` — our v3 welcomes and GCE commits never apply, so every cross-client group flow dies at welcome processing. We still parse v3 happily on the way in; we just don't emit it until mdk publishes the forward-compat fix MIP-01 mandates. * MarmotManager.updateGroupMetadata: MERGE extensions instead of REPLACING. RFC 9420 §12.1.7 says GCE proposals blow away the old extension list; callers that pass only [marmot_group_data] dropped [required_capabilities], which peers then reject. Preserve every slot except the one we're updating. * MarmotManager.createGroup: new optional `initialMetadata` parameter that bakes MarmotGroupData into epoch-0 GroupContext.extensions directly. Without it, creators had to publish a pre-membership "bootstrap" commit that no later joiner could decrypt — each such peer then burned their retry budget on an undecryptable kind:445 before seeing the real state. Threaded through MlsGroup.create / MlsGroupManager.createGroup. * MarmotManager.mlsGroupIdHex: new translation helper so any code juggling the MIP-01 nostr_group_id (what amy indexes on) and the MLS GroupContext groupId (what mdk indexes on) can cross-reference them without reaching into MlsGroupManager directly. amethyst module --------------- * Account.leaveMarmotGroup: self-demote before SelfRemove per MIP-01, and promote a surviving member to admin first if the caller is the sole admin (otherwise we'd throw "admin depletion"). Matches the cli/GroupMembershipCommands.leave flow. cli (amy) --------- * Context.syncIncoming: don't advance `giftWrapSince` on empty polls (so the first-ever sync doesn't bump the cursor past every past-timestamped wrap we've ever been sent), subtract 2 days lookback when filtering (NIP-59 randomWithTwoDays gift wraps can have any createdAt in the last 48h), and only advance `groupSince` for groups we actually received events for. * Context.syncIncoming: after ingest, if any Welcome consumed a KeyPackage, rotate and publish a fresh one immediately. MIP-00 requires this — a KP can only be welcomed once and leaving the consumed one on relays just means later senders invite us with a bundle we no longer have private keys for. * Context.resolveGroupId: accept either nostr_group_id (amy's primary key) or the MLS GroupContext groupId (what wn emits) on every verb that takes a group id. Wired through GroupAdd/Remove/Leave/Metadata /Read, Message send/list, and all the await* verbs so harness scripts never have to juggle both forms for a single group. * GroupCreateCommand: bake initial metadata into epoch 0 (see the quartz change above). Dropped the now-redundant bootstrap commit publish and tightened the JSON output to include `mls_group_id`. * GroupMembershipCommands.leave: self-demote admin before SelfRemove; promote an heir if we're the only admin, otherwise the GCE would deplete admins and the leave aborts. * MarmotIngest.ingestGiftWrap: unwrap the sealed-rumor layer. NIP-59 wrap is gift-wrap(kind:1059) → seal(kind:13) → rumor; the old code only unwrapped once and then checked `inner.kind == 444`, which is always false because inner is actually the seal. Unseal once more before the Welcome check. This single fix is what unsticks every amy-side Welcome ingestion. wn harness patches + scripts ---------------------------- * whitenoise-defaults-env.patch: honour $WHITENOISE_DISCOVERY_RELAYS in `Relay::defaults()` (release builds otherwise bake damus.io / primal / nos.lol into every new account's NIP-65 / Inbox / KeyPackage lists, which breaks publishing and prevents the inbox subscription plane from ever reaching an operational state in a sandbox). * setup.sh: sleep 2s after amy's initial kind:30443 publish so nostr-rs-relay has a chance to fsync before wn's first targeted discovery query. Without it wn's `keys check` races the relay's WAL flush and intermittently returns NotFound. * lib.sh: peel wn's `{"result": …}` wrapper in `jq_group_id`, `wait_for_invite`, `wait_for_message`, `wait_for_member`. Post-v0.2 wn `--json` output nests everything under `.result` (and `groups invites[]` nests further under `.group.mls_group_id`) — these helpers were still pattern-matching on the flat shape, so they returned empty strings for a perfectly good response. * tests-{create,manage,extras}.sh: track both group IDs per test (amy's nostr + wn's MLS), pass each CLI the id it understands, and bump the post-commit wait timeouts to 90–120s so wn's exponential retry backoff has time to work through the pre-membership commits it can't decrypt and get to the ones it can. https://claude.ai/code/session_016kAxdp6ubB5CnF9URhCEzP	2026-04-22 00:28:45 +00:00
Claude	37515ec975	feat(marmot-interop): let wnd run in sandboxes without kernel keyring Containers / CI often block the keyutils syscalls wnd uses by default to store secret keys (add_key returns EACCES, keyctl_search returns ENOSYS). The integration-tests feature ships a mock keyring store, but the stock wnd binary doesn't wire it up. Add a tiny opt-in patch. Changes: - headless/patches/whitenoise-mock-keyring.patch: if built with the integration-tests feature and \$WHITENOISE_MOCK_KEYRING is set, wnd calls Whitenoise::initialize_mock_keyring_store() before the normal init path. Harmless on production builds. - headless/setup.sh: apply both harness patches generically from a list; build wn/wnd with --features cli,integration-tests; export WHITENOISE_MOCK_KEYRING=1 alongside WHITENOISE_DISCOVERY_RELAYS when launching each daemon. - preflight swaps keyctl for patch in required tools — we no longer need a real session keyring. https://claude.ai/code/session_01M6dCKAF5Y1VyHGZPjzwDXq	2026-04-21 22:20:54 +00:00
Claude	6213fb2197	feat(marmot-interop): patch wnd to honour \$WHITENOISE_DISCOVERY_RELAYS Without this, wnd exits on startup with NoRelayConnections in any environment that can't reach wnd's baked-in public discovery relay set (nos.lol, relay.damus.io, …). The headless harness runs entirely on a loopback relay, so hitting the public internet is never OK. Changes: - headless/patches/whitenoise-discovery-env.patch: adds an env-var override at the top of DiscoveryPlaneConfig::curated_default_relays. When \$WHITENOISE_DISCOVERY_RELAYS is set (comma-separated) we use that list instead of the hardcoded public one. - headless/setup.sh: applies the patch in preflight (idempotent — checks for a .headless-discovery-patched marker), invalidates the previous wn/wnd build on first patch, rebuilds, and threads \$WHITENOISE_DISCOVERY_RELAYS=\$RELAY_URL into every start_daemon invocation. https://claude.ai/code/session_01M6dCKAF5Y1VyHGZPjzwDXq	2026-04-21 22:14:16 +00:00

Author

SHA1

Message

Date

Claude

f08b010f50

fix(marmot,cli,interop): interop-compatible Marmot flows and harness correctness

Five protocol-level fixes and a batch of harness correctness fixes to get
the headless Marmot/Whitenoise interop harness from 1/13 to 5/13 passing
cleanly, with the remaining failures all rooted in wn's per-account
serial event-processor retry backoff (which drops undecryptable
pre-membership commits after several minutes) rather than amy behaviour.

quartz + commons
----------------
* MarmotGroupData: hold CURRENT_VERSION at 2. mdk-core (the Rust MLS
  engine used by whitenoise-rs) strict-rejects v3 payloads with
  `ExtensionFormatError("Trailing bytes in NostrGroupDataExtension")`
  — our v3 welcomes and GCE commits never apply, so every cross-client
  group flow dies at welcome processing. We still parse v3 happily on
  the way in; we just don't emit it until mdk publishes the
  forward-compat fix MIP-01 mandates.
* MarmotManager.updateGroupMetadata: MERGE extensions instead of
  REPLACING. RFC 9420 §12.1.7 says GCE proposals blow away the old
  extension list; callers that pass only [marmot_group_data] dropped
  [required_capabilities], which peers then reject. Preserve every slot
  except the one we're updating.
* MarmotManager.createGroup: new optional `initialMetadata` parameter
  that bakes MarmotGroupData into epoch-0 GroupContext.extensions
  directly. Without it, creators had to publish a pre-membership
  "bootstrap" commit that no later joiner could decrypt — each such
  peer then burned their retry budget on an undecryptable kind:445
  before seeing the real state. Threaded through MlsGroup.create /
  MlsGroupManager.createGroup.
* MarmotManager.mlsGroupIdHex: new translation helper so any code
  juggling the MIP-01 nostr_group_id (what amy indexes on) and the
  MLS GroupContext groupId (what mdk indexes on) can cross-reference
  them without reaching into MlsGroupManager directly.

amethyst module
---------------
* Account.leaveMarmotGroup: self-demote before SelfRemove per MIP-01,
  and promote a surviving member to admin first if the caller is the
  sole admin (otherwise we'd throw "admin depletion"). Matches the
  cli/GroupMembershipCommands.leave flow.

cli (amy)
---------
* Context.syncIncoming: don't advance `giftWrapSince` on empty polls
  (so the first-ever sync doesn't bump the cursor past every
  past-timestamped wrap we've ever been sent), subtract 2 days lookback
  when filtering (NIP-59 randomWithTwoDays gift wraps can have any
  createdAt in the last 48h), and only advance `groupSince` for groups
  we actually received events for.
* Context.syncIncoming: after ingest, if any Welcome consumed a
  KeyPackage, rotate and publish a fresh one immediately. MIP-00
  requires this — a KP can only be welcomed once and leaving the
  consumed one on relays just means later senders invite us with a
  bundle we no longer have private keys for.
* Context.resolveGroupId: accept either nostr_group_id (amy's primary
  key) or the MLS GroupContext groupId (what wn emits) on every verb
  that takes a group id. Wired through GroupAdd/Remove/Leave/Metadata
  /Read, Message send/list, and all the await* verbs so harness
  scripts never have to juggle both forms for a single group.
* GroupCreateCommand: bake initial metadata into epoch 0 (see the
  quartz change above). Dropped the now-redundant bootstrap commit
  publish and tightened the JSON output to include `mls_group_id`.
* GroupMembershipCommands.leave: self-demote admin before SelfRemove;
  promote an heir if we're the only admin, otherwise the GCE would
  deplete admins and the leave aborts.
* MarmotIngest.ingestGiftWrap: unwrap the sealed-rumor layer. NIP-59
  wrap is gift-wrap(kind:1059) → seal(kind:13) → rumor; the old code
  only unwrapped once and then checked `inner.kind == 444`, which is
  always false because inner is actually the seal. Unseal once more
  before the Welcome check. This single fix is what unsticks every
  amy-side Welcome ingestion.

wn harness patches + scripts
----------------------------
* whitenoise-defaults-env.patch: honour $WHITENOISE_DISCOVERY_RELAYS in
  `Relay::defaults()` (release builds otherwise bake damus.io / primal
  / nos.lol into every new account's NIP-65 / Inbox / KeyPackage
  lists, which breaks publishing and prevents the inbox subscription
  plane from ever reaching an operational state in a sandbox).
* setup.sh: sleep 2s after amy's initial kind:30443 publish so
  nostr-rs-relay has a chance to fsync before wn's first targeted
  discovery query. Without it wn's `keys check` races the relay's
  WAL flush and intermittently returns NotFound.
* lib.sh: peel wn's `{"result": …}` wrapper in `jq_group_id`,
  `wait_for_invite`, `wait_for_message`, `wait_for_member`. Post-v0.2
  wn `--json` output nests everything under `.result` (and
  `groups invites[]` nests further under `.group.mls_group_id`) —
  these helpers were still pattern-matching on the flat shape, so
  they returned empty strings for a perfectly good response.
* tests-{create,manage,extras}.sh: track both group IDs per test
  (amy's nostr + wn's MLS), pass each CLI the id it understands, and
  bump the post-commit wait timeouts to 90–120s so wn's exponential
  retry backoff has time to work through the pre-membership commits
  it can't decrypt and get to the ones it can.

https://claude.ai/code/session_016kAxdp6ubB5CnF9URhCEzP

2026-04-22 00:28:45 +00:00

Claude

37515ec975

feat(marmot-interop): let wnd run in sandboxes without kernel keyring

Containers / CI often block the keyutils syscalls wnd uses by default
to store secret keys (add_key returns EACCES, keyctl_search returns
ENOSYS). The integration-tests feature ships a mock keyring store, but
the stock wnd binary doesn't wire it up. Add a tiny opt-in patch.

Changes:
- headless/patches/whitenoise-mock-keyring.patch: if built with the
  integration-tests feature and \$WHITENOISE_MOCK_KEYRING is set, wnd
  calls Whitenoise::initialize_mock_keyring_store() before the normal
  init path. Harmless on production builds.
- headless/setup.sh: apply both harness patches generically from a
  list; build wn/wnd with --features cli,integration-tests; export
  WHITENOISE_MOCK_KEYRING=1 alongside WHITENOISE_DISCOVERY_RELAYS when
  launching each daemon.
- preflight swaps keyctl for patch in required tools — we no longer
  need a real session keyring.

https://claude.ai/code/session_01M6dCKAF5Y1VyHGZPjzwDXq

2026-04-21 22:20:54 +00:00

Claude

6213fb2197

feat(marmot-interop): patch wnd to honour \$WHITENOISE_DISCOVERY_RELAYS

Without this, wnd exits on startup with NoRelayConnections in any
environment that can't reach wnd's baked-in public discovery relay
set (nos.lol, relay.damus.io, …). The headless harness runs entirely
on a loopback relay, so hitting the public internet is never OK.

Changes:
- headless/patches/whitenoise-discovery-env.patch: adds an env-var
  override at the top of DiscoveryPlaneConfig::curated_default_relays.
  When \$WHITENOISE_DISCOVERY_RELAYS is set (comma-separated) we use
  that list instead of the hardcoded public one.
- headless/setup.sh: applies the patch in preflight (idempotent —
  checks for a .headless-discovery-patched marker), invalidates the
  previous wn/wnd build on first patch, rebuilds, and threads
  \$WHITENOISE_DISCOVERY_RELAYS=\$RELAY_URL into every start_daemon
  invocation.

https://claude.ai/code/session_01M6dCKAF5Y1VyHGZPjzwDXq

2026-04-21 22:14:16 +00:00

3 Commits