From 1edbe81b6087eafe5c658ce94c8795df6539868b Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 23 Apr 2026 17:20:57 +0000 Subject: [PATCH 01/11] docs(cli): plan NIP-17 DM send/list/await verbs Design doc for `amy dm send|list|await`, reusing the existing Quartz gift-wrap (NIP-17/NIP-59/NIP-44) pipeline. Single file under cli/plans/; extracts FilterGiftWrapsToPubkey from amethyst/ to commons/ as a prerequisite commit so the CLI can depend on it without pulling in :amethyst. --- cli/plans/2026-04-23-nip17-dm.md | 252 +++++++++++++++++++++++++++++++ 1 file changed, 252 insertions(+) create mode 100644 cli/plans/2026-04-23-nip17-dm.md diff --git a/cli/plans/2026-04-23-nip17-dm.md b/cli/plans/2026-04-23-nip17-dm.md new file mode 100644 index 000000000..9d69cfaa2 --- /dev/null +++ b/cli/plans/2026-04-23-nip17-dm.md @@ -0,0 +1,252 @@ +# NIP-17 DMs in `amy` + +**Status:** plan · **Date:** 2026-04-23 · **Roadmap row:** +`cli/ROADMAP.md:55` ("NIP-17 DMs send / list / read") · **Order:** +step 5 in `cli/ROADMAP.md:91`. + +Adds three verbs: + +``` +amy dm send [--peer ...] +amy dm list [--peer ] [--since ] [--limit N] [--timeout SECS] +amy dm await --peer --match [--timeout SECS] +``` + +`` accepts everything `Context.requireUserHex` does: +`npub`, `nprofile`, 64-hex, `name@domain.tld`. `--peer` may repeat +on `send` to start a multi-recipient chat (NIP-17 supports group +DMs); on `list`/`await` it filters by counter-party. + +--- + +## Guiding principles + +1. **Thin assembly only.** Every protocol piece — sealing, wrapping, + NIP-44 — already lives in `quartz/`. The CLI just orchestrates. +2. **One JSON line out, snake_case keys, exit codes 0/1/2/124.** +3. **Reuse Marmot's gift-wrap plumbing** where it already exists in + `Context` (cursor, drain, publish-and-confirm). +4. **No business logic in `cli/`.** The single piece of Android-side + code we depend on (`filterGiftWrapsToPubkey`) is extracted into + `commons/` first, in its own commit. + +--- + +## API surface we reuse (no changes needed) + +| Concern | Class | File | +|---|---|---| +| Build `kind:14` chat message template | `ChatMessageEvent.build(msg, to, …)` | `quartz/.../nip17Dm/messages/ChatMessageEvent.kt:51` | +| Sign + seal + wrap (one wrap per recipient incl. self) | `NIP17Factory().createMessageNIP17(template, signer)` | `quartz/.../nip17Dm/NIP17Factory.kt:37` | +| `GiftWrapEvent.create / unwrapOrNull / recipientPubKey` | `GiftWrapEvent` (kind 1059) | `quartz/.../nip59Giftwrap/wraps/GiftWrapEvent.kt` | +| Inner seal layer (kind 13) | `SealedRumorEvent` | `quartz/.../nip59Giftwrap/seals/SealedRumorEvent.kt` | +| NIP-44 v2 encrypt/decrypt (called from wrap/seal) | `Nip44` singleton | `quartz/.../nip44Encryption/Nip44.kt` | +| Resolve recipient relay lists (10050 / 10051 / 10002) | `RecipientRelayFetcher.fetchRelayLists` | `quartz/.../marmot/RecipientRelayFetcher.kt:80` | +| Publish + per-relay ACK | `Context.publish(event, relays)` | `cli/.../Context.kt:154` | +| Drain a subscription until EOSE/timeout | `Context.drain(filters, timeoutMs)` | `cli/.../Context.kt:168` | +| Resolve identifiers | `Context.requireUserHex(input)` | `cli/.../Context.kt:117` | +| In-memory signer (NIP-44 ops included) | `NostrSignerInternal` (`Context.signer`) | `cli/.../Context.kt:68` | + +Everything above is already on the JVM classpath of the `cli` module. + +--- + +## Extraction matrix + +| Piece | Status | Current | Target | Notes | +|---|---|---|---|---| +| `NIP17Factory`, `GiftWrapEvent`, `SealedRumorEvent`, `Nip44`, `ChatMessageEvent` | ✅ Reuse | `quartz/` | — | Pure protocol; CLI calls directly. | +| `RecipientRelayFetcher.fetchRelayLists` | ✅ Reuse | `quartz/.../marmot/` | — | Already used by `AwaitCommands.awaitKeyPackage`; same call shape. | +| `filterGiftWrapsToPubkey` | 📦 Extract | `amethyst/.../service/relayClient/reqCommand/account/nip59GiftWraps/FilterGiftWrapsToPubkey.kt:31` | `commons/.../relayClient/nip17Dm/FilterGiftWrapsToPubkey.kt` | Pure function, no Android deps. Update Android caller (`AccountGiftWrapsEoseManager.kt:55`) in the same commit. | +| Per-recipient relay routing (kind:10050 → 10002 read → bootstrap) | 🆕 New (in CLI) | inline inside `Account.computeRelayListToBroadcast` (`amethyst/.../model/Account.kt:908`) | `cli/.../commands/DmCommands.kt` (private helper) | Tiny — three-line fallback. Promoting it to commons can wait until a second consumer appears. | +| Send command | 🆕 New | — | `cli/.../commands/DmCommands.kt :: send()` | | +| List command | 🆕 New | — | `cli/.../commands/DmCommands.kt :: list()` | | +| Await command | 🆕 New | — | `cli/.../commands/DmCommands.kt :: await()` | | +| Cursor on disk | ✅ Reuse | `RunState.giftWrapSince` (already used by Marmot `Context.syncIncoming`) | — | Don't fork. DM drain advances the same cursor only when used by `dm list` without `--peer/--since`. | +| Routing in `Main.kt` / `Commands.kt` | 🆕 New | — | `Main.kt` dispatch + `Commands.dm(...)` | One new branch each. | +| Account-bootstrap of own DM inbox (kind:10050) | ⚠️ Not in this plan | `amethyst/.../model/AccountSettings.kt:668` | future `amy relay publish-lists --dm` | Out of scope; we read existing inbox from disk and warn if missing. | + +**Rule check (from `amy-expert`):** +- *Rule 1 (thin)*: ✅ — only one new file in `cli/commands/`, < 200 lines. +- *Rule 2 (JSON)*: ✅ — see Output Contract below. +- *Rule 3 (non-interactive)*: ✅ — `--timeout` on the wait verb, no prompts. +- *Rule 4 (data-dir is the world)*: ✅ — relays + signer + cursor all reload from `--data-dir`. +- *Rule 5 (extract first)*: applies once, to `filterGiftWrapsToPubkey` (separate commit). + +--- + +## Send flow (`amy dm send`) + +``` +parse argv → Context.open → ctx.prepare() + ↓ +recipients = (positional + --peer) → requireUserHex each + ↓ +template = ChatMessageEvent.build(text, recipients.map { PTag(it) }) + ↓ +result = NIP17Factory().createMessageNIP17(template, ctx.signer) + ─ signs the inner kind:14 + ─ produces one GiftWrapEvent (1059) per recipient INCLUDING self + ↓ +for each wrap in result.wraps: + relayList = recipientDmRelays(wrap.recipientPubKey()) + ─ 1. RecipientRelayFetcher.fetchRelayLists(client, recipient, ctx.bootstrapRelays()) + ─ 2. .dmInboxOrFallback() (kind:10050 → kind:10002 read marker) + ─ 3. fallback to ctx.bootstrapRelays() if both empty + ack = ctx.publish(wrap, relayList) + record { recipient, wrap_id, published_to: ack.true-keys } + ↓ +emit one JSON line: + { "event_id": result.msg.id, # inner kind:14 id (stable identity of the message) + "kind": 14, + "recipients": [ {"pubkey":…, "wrap_id":…, "published_to":[…], "relays_tried":[…]} … ] } +``` + +Notes: +- `result.wraps` already includes a self-addressed wrap; we publish it + the same way as the others so `amy dm list` works as a sanity check + immediately after sending. +- We do NOT add the wrap to any local cache — there is no `LocalCache` + in the CLI; on next `amy dm list` we re-pull from relays. +- If `recipientDmRelays` returns empty for a recipient, exit code `1` + with `{"error":"no_dm_relays","detail":""}`. Do not silently drop. + +--- + +## List flow (`amy dm list`) + +``` +parse argv → Context.open → ctx.prepare() + ↓ +inbox = ctx.relays.normalized("inbox") + .ifEmpty { ctx.outboxRelays() } + .ifEmpty { ctx.bootstrapRelays() } + ↓ +since = --since OR (cursor advance mode: state.giftWrapSince OR null) +filters = inbox.flatMap { filterGiftWrapsToPubkey(it, ctx.signer.pubKey, since) } + .groupBy { it.relay }.mapValues { it.value.map(RelayBasedFilter::filter) } + ↓ +raw = ctx.drain(filters, timeoutMs = --timeout * 1000 ?: 8_000) + ↓ +messages = raw.mapNotNull { (relay, event) -> + if (event !is GiftWrapEvent) return@mapNotNull null + val sealed = event.unwrapOrNull(ctx.signer) as? SealedRumorEvent ?: return@mapNotNull null + val inner = sealed.unsealOrNull(ctx.signer) as? ChatMessageEvent ?: return@mapNotNull null + DecryptedMsg(inner, relay, event.id) +}.dedupBy { it.inner.id } + .filter { peerHex == null || peerHex in it.inner.groupMembers() } + .sortedBy { it.inner.createdAt } + .takeLast(limit) + ↓ +emit: + { "peer": , "messages": [ + { "id": , "wrap_id": <1059 id>, + "from": , "to": [...], + "content": , "created_at": <unix>, + "relay": <where wrap was found> }, … ] } +``` + +Cursor handling: only when `--since` and `--peer` are both absent do we +advance `state.giftWrapSince` to `now()` after a successful drain. +Otherwise the call is a stateless query (parity with how Marmot reads +behave). `GIFT_WRAP_LOOKBACK_SECS` (already defined in `Context.kt`) is +applied automatically by `filterGiftWrapsToPubkey` (it does +`since - twoDays()` itself). + +--- + +## Await flow (`amy dm await`) + +Mirror `AwaitCommands.awaitMessage` line-for-line +(`cli/.../commands/AwaitCommands.kt`). Difference: matches against the +unwrapped `ChatMessageEvent.content` instead of a Marmot inner event. +Polls every 2 s, decrypts what arrived, returns the first match or +exits `124`. + +```json +{ "id": "<kind14 id>", "from": "<pubkey>", "content": "<text>", + "created_at": <unix>, "wrap_id": "<1059 id>", "relay": "<url>" } +``` + +--- + +## Output contract (stable JSON keys) + +| Verb | Success keys | +|---|---| +| `dm send` | `event_id` (kind 14), `kind`, `recipients[]{pubkey, wrap_id, published_to[], relays_tried[]}` | +| `dm list` | `peer` (hex or null), `messages[]{id, wrap_id, from, to[], content, created_at, relay}` | +| `dm await` | `id, from, content, created_at, wrap_id, relay` | + +Errors (stderr, single-line): +- `bad_args` — missing positional / unparseable `npub` +- `no_dm_relays` — recipient has no kind:10050 and no NIP-65 read +- `no_inbox_relays` — local user has no inbox configured AND no bootstrap +- `runtime` — anything else from `Context`/quartz +- `timeout` (124) — only from `dm await` + +Adding a new key later is non-breaking; renaming/removing requires a +note in the commit message. + +--- + +## File layout after the change + +``` +cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/ +├── Main.kt # + "dm" branch → Commands.dm(…) +├── commands/ +│ ├── Commands.kt # + suspend fun dm(dataDir, tail) +│ └── DmCommands.kt # NEW: object DmCommands { dispatch, send, list, await } +└── (nothing else) + +commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/relayClient/nip17Dm/ +└── FilterGiftWrapsToPubkey.kt # NEW (extracted from amethyst/) + +amethyst/src/main/java/com/vitorpamplona/amethyst/service/relayClient/reqCommand/account/nip59GiftWraps/ +├── AccountGiftWrapsEoseManager.kt # import-only update to point at commons +└── FilterGiftWrapsToPubkey.kt # DELETED in the same commit +``` + +`README.md` table + `ROADMAP.md` row update in the same PR. + +--- + +## Implementation sequence + +Each bullet = one commit. + +1. **Extract** `filterGiftWrapsToPubkey` from `amethyst/` to + `commons/.../relayClient/nip17Dm/` and re-point the Android caller. + Run `./gradlew :amethyst:compileDebugKotlin :commons:build`. +2. **Add `amy dm send`** — wires `NIP17Factory` + per-recipient relay + resolution + publish. Manual smoke test: send to self, then to a + second test identity in a separate `--data-dir`. +3. **Add `amy dm list`** — drain + unwrap + JSON. Smoke test reads back + the message published in step 2. +4. **Add `amy dm await`** — polling variant, copy of + `AwaitCommands.awaitMessage` shape. +5. **Docs** — add three rows to `cli/README.md` command table; flip + `cli/ROADMAP.md:55` from 🆕 → ✅; record the JSON shape in the + README's "JSON output" section. +6. **`./gradlew spotlessApply`** before each commit. + +--- + +## Open questions / non-goals + +- **Publishing our own kind:10050.** Out of scope; assume the user has + one already (created via Amethyst, or via a future + `amy relay publish-lists --dm`). If missing we still send (using + bootstrap as fallback) but do not auto-publish a list. +- **Reactions / replies / encrypted file attachments (kind 32 with + NIP-17 attachment metadata).** Out of scope — separate plan, if ever. +- **NIP-04 legacy DMs (kind 4).** Not supported. NIP-17-only is a + deliberate choice consistent with `nostr-expert` guidance. +- **Persistent local cache of decrypted messages.** Not in this plan. + Each `amy dm list` re-decrypts; identity-equivalence is by inner + kind:14 id so it remains idempotent. If this becomes a perf issue we + add a `dms/<peer>.log` store mirroring Marmot's pattern — separate + commit. +- **Live subscription (`--watch`).** Out of scope; CLI is one-shot by + Rule 3. From c43a69b083f03646e9326d315e043a5079d1fd46 Mon Sep 17 00:00:00 2001 From: Claude <noreply@anthropic.com> Date: Thu, 23 Apr 2026 18:16:25 +0000 Subject: [PATCH 02/11] refactor: move filterGiftWrapsToPubkey from amethyst to commons MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Pure function with no Android dependencies — relocating it to commons/relayClient/nip17Dm/ so cli/ can reuse it for NIP-17 DM subscriptions without pulling in :amethyst. Android caller updated to import from the new package. Prep for `amy dm send|list|await` — see cli/plans/2026-04-23-nip17-dm.md. --- .../account/nip59GiftWraps/AccountGiftWrapsEoseManager.kt | 1 + .../commons/relayClient/nip17Dm}/FilterGiftWrapsToPubkey.kt | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) rename {amethyst/src/main/java/com/vitorpamplona/amethyst/service/relayClient/reqCommand/account/nip59GiftWraps => commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/relayClient/nip17Dm}/FilterGiftWrapsToPubkey.kt (95%) diff --git a/amethyst/src/main/java/com/vitorpamplona/amethyst/service/relayClient/reqCommand/account/nip59GiftWraps/AccountGiftWrapsEoseManager.kt b/amethyst/src/main/java/com/vitorpamplona/amethyst/service/relayClient/reqCommand/account/nip59GiftWraps/AccountGiftWrapsEoseManager.kt index 54b9e0fd4..8f5484a1a 100644 --- a/amethyst/src/main/java/com/vitorpamplona/amethyst/service/relayClient/reqCommand/account/nip59GiftWraps/AccountGiftWrapsEoseManager.kt +++ b/amethyst/src/main/java/com/vitorpamplona/amethyst/service/relayClient/reqCommand/account/nip59GiftWraps/AccountGiftWrapsEoseManager.kt @@ -20,6 +20,7 @@ */ package com.vitorpamplona.amethyst.service.relayClient.reqCommand.account.nip59GiftWraps +import com.vitorpamplona.amethyst.commons.relayClient.nip17Dm.filterGiftWrapsToPubkey import com.vitorpamplona.amethyst.model.User import com.vitorpamplona.amethyst.service.relayClient.eoseManagers.PerUserEoseManager import com.vitorpamplona.amethyst.service.relayClient.reqCommand.account.AccountQueryState diff --git a/amethyst/src/main/java/com/vitorpamplona/amethyst/service/relayClient/reqCommand/account/nip59GiftWraps/FilterGiftWrapsToPubkey.kt b/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/relayClient/nip17Dm/FilterGiftWrapsToPubkey.kt similarity index 95% rename from amethyst/src/main/java/com/vitorpamplona/amethyst/service/relayClient/reqCommand/account/nip59GiftWraps/FilterGiftWrapsToPubkey.kt rename to commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/relayClient/nip17Dm/FilterGiftWrapsToPubkey.kt index f8e1e3f26..191d4e82e 100644 --- a/amethyst/src/main/java/com/vitorpamplona/amethyst/service/relayClient/reqCommand/account/nip59GiftWraps/FilterGiftWrapsToPubkey.kt +++ b/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/relayClient/nip17Dm/FilterGiftWrapsToPubkey.kt @@ -18,7 +18,7 @@ * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ -package com.vitorpamplona.amethyst.service.relayClient.reqCommand.account.nip59GiftWraps +package com.vitorpamplona.amethyst.commons.relayClient.nip17Dm import com.vitorpamplona.quartz.nip01Core.core.HexKey import com.vitorpamplona.quartz.nip01Core.relay.client.pool.RelayBasedFilter From 15205c24b5cae68621c5e2f8e766c41fbc3b9008 Mon Sep 17 00:00:00 2001 From: Claude <noreply@anthropic.com> Date: Thu, 23 Apr 2026 18:23:12 +0000 Subject: [PATCH 03/11] feat(cli): amy dm send|list|await (NIP-17) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds three verbs wired into the top-level dispatch: - `amy dm send RECIPIENT TEXT` — builds a kind:14 ChatMessageEvent, runs it through NIP17Factory to seal and gift-wrap, then publishes each wrap to its recipient's DM inbox (kind:10050, fallback kind:10002 read, final fallback bootstrap). Emits one JSON line with the inner kind:14 id, per-recipient wrap ids, and per-relay ACK status. - `amy dm list [--peer NPUB] [--since TS] [--limit N] [--timeout SECS]` — drains gift wraps addressed to us from our inbox relays (or outbox / bootstrap fallbacks), unwraps+unseals each one, dedupes by inner id, and returns them oldest-first. With no flags it advances the giftWrapSince cursor in state.json (matches the Marmot syncIncoming convention); with --peer/--since it's a stateless query. - `amy dm await --peer NPUB --match TEXT [--timeout SECS]` — polls the same drain on a 2s tick until a DM from NPUB contains TEXT. Timeout exits 124 like the other await verbs. All three reuse Quartz entirely (NIP17Factory, GiftWrapEvent, SealedRumorEvent, RecipientRelayFetcher); the only shared piece we needed in commons — filterGiftWrapsToPubkey — was extracted in the previous commit. No business logic leaks into cli/. Plan: cli/plans/2026-04-23-nip17-dm.md. --- cli/README.md | 3 + cli/ROADMAP.md | 2 +- .../com/vitorpamplona/amethyst/cli/Main.kt | 11 + .../amethyst/cli/commands/Commands.kt | 5 + .../amethyst/cli/commands/DmCommands.kt | 280 ++++++++++++++++++ 5 files changed, 300 insertions(+), 1 deletion(-) create mode 100644 cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/DmCommands.kt diff --git a/cli/README.md b/cli/README.md index 036071d29..14450c09d 100644 --- a/cli/README.md +++ b/cli/README.md @@ -137,6 +137,9 @@ Run `amy --help` for the canonical list. As of today: | `marmot await message GID --match TEXT` | Block until a message containing `TEXT` lands. | | `marmot await rename GID --name NAME` | Block until GID's name matches. | | `marmot await epoch GID --min N` | Block until GID's MLS epoch is ≥ N. | +| `dm send RECIPIENT TEXT` | Send a NIP-17 gift-wrapped DM (kind:14 inside kind:1059). Publishes to the recipient's kind:10050 → kind:10002 read → our bootstrap pool. | +| `dm list [--peer NPUB] [--since TS] [--limit N] [--timeout SECS]` | Drain and decrypt gift wraps on our inbox relays. With neither `--peer` nor `--since` the gift-wrap cursor in `state.json` is advanced to the newest message seen. | +| `dm await --peer NPUB --match TEXT [--timeout SECS]` | Block until a DM from NPUB containing TEXT arrives. Timeout exits 124. | All `await` verbs accept `--timeout SECS` (default 30). Timeout exits 124 so scripts can distinguish "condition never happened" from "the command diff --git a/cli/ROADMAP.md b/cli/ROADMAP.md index fe42a3f40..466cd8410 100644 --- a/cli/ROADMAP.md +++ b/cli/ROADMAP.md @@ -52,7 +52,7 @@ Status legend: ✅ shipped · 📦 logic lives in `commons/`, needs a command · | NIP-01 feed read (`amy feed home`, `amy feed hashtag #X`, `amy feed profile NPUB`) | 🆕 | Extract `FeedFilter` usage from `amethyst/ui/dal/` into `commons/` entry points. | | NIP-02 follow list add / remove / list | 🆕 | Logic in `amethyst/model/nip02FollowLists/`. | | NIP-09 event deletion | 🆕 | Builder exists in quartz. | -| NIP-17 DMs send / list / read | 🆕 | Gift-wrap path is already in `commons/` via Marmot — generalise. | +| NIP-17 DMs send / list / await | ✅ | `DmCommands` — reuses Quartz `NIP17Factory` + `RecipientRelayFetcher`; filter extracted to `commons/relayClient/nip17Dm/`. Plan: [`cli/plans/2026-04-23-nip17-dm.md`](./plans/2026-04-23-nip17-dm.md). | | NIP-18 reposts / quotes | 🆕 | | | NIP-25 reactions | 🆕 | | | NIP-51 lists (bookmarks, mute, follow sets) | 🆕 | `amethyst/model/nip51Lists/` | diff --git a/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/Main.kt b/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/Main.kt index d9cf303bb..dd8d4dec4 100644 --- a/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/Main.kt +++ b/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/Main.kt @@ -127,6 +127,10 @@ private suspend fun dispatch(argv: Array<String>): Int { marmotDispatch(dataDir, tail) } + "dm" -> { + Commands.dm(dataDir, tail) + } + else -> { System.err.println("unknown subcommand: $head") printUsage() @@ -189,6 +193,13 @@ private fun printUsage() { | relay list print configured relays | relay publish-lists publish kind:10002 + kind:10050 | + |Direct messages (NIP-17): + | dm send RECIPIENT TEXT send a gift-wrapped DM + | dm list [--peer NPUB] [--since TS] list decrypted DMs (advances cursor + | [--limit N] [--timeout SECS] only when no flags are passed) + | dm await --peer NPUB --match TEXT wait for a matching DM + | [--timeout SECS] (default 30s, exit 124 on timeout) + | |Marmot (MLS group messaging): | marmot key-package publish publish a fresh KeyPackage | marmot key-package check NPUB fetch NPUB's KeyPackage from relays diff --git a/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/Commands.kt b/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/Commands.kt index 4b287ce96..aacf74d04 100644 --- a/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/Commands.kt +++ b/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/Commands.kt @@ -70,4 +70,9 @@ object Commands { dataDir: DataDir, tail: Array<String>, ): Int = AwaitCommands.dispatch(dataDir, tail) + + suspend fun dm( + dataDir: DataDir, + tail: Array<String>, + ): Int = DmCommands.dispatch(dataDir, tail) } diff --git a/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/DmCommands.kt b/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/DmCommands.kt new file mode 100644 index 000000000..287d9c29a --- /dev/null +++ b/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/DmCommands.kt @@ -0,0 +1,280 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.amethyst.cli.commands + +import com.vitorpamplona.amethyst.cli.Args +import com.vitorpamplona.amethyst.cli.AwaitTimeout +import com.vitorpamplona.amethyst.cli.Context +import com.vitorpamplona.amethyst.cli.DataDir +import com.vitorpamplona.amethyst.cli.Json +import com.vitorpamplona.amethyst.commons.relayClient.nip17Dm.filterGiftWrapsToPubkey +import com.vitorpamplona.quartz.marmot.RecipientRelayFetcher +import com.vitorpamplona.quartz.nip01Core.core.HexKey +import com.vitorpamplona.quartz.nip01Core.relay.normalizer.NormalizedRelayUrl +import com.vitorpamplona.quartz.nip01Core.tags.people.PTag +import com.vitorpamplona.quartz.nip17Dm.NIP17Factory +import com.vitorpamplona.quartz.nip17Dm.messages.ChatMessageEvent +import com.vitorpamplona.quartz.nip59Giftwrap.seals.SealedRumorEvent +import com.vitorpamplona.quartz.nip59Giftwrap.wraps.GiftWrapEvent +import kotlinx.coroutines.delay + +/** + * NIP-17 direct-message verbs. All three reuse the Quartz gift-wrap pipeline + * (NIP-17 chat message → NIP-59 seal → NIP-59 gift wrap via NIP-44) — this + * command file is pure orchestration. + */ +object DmCommands { + suspend fun dispatch( + dataDir: DataDir, + tail: Array<String>, + ): Int { + if (tail.isEmpty()) return Json.error("bad_args", "dm <send|list|await> …") + val rest = tail.drop(1).toTypedArray() + return when (tail[0]) { + "send" -> send(dataDir, rest) + "list" -> list(dataDir, rest) + "await" -> await(dataDir, rest) + else -> Json.error("bad_args", "dm ${tail[0]}") + } + } + + private suspend fun send( + dataDir: DataDir, + rest: Array<String>, + ): Int { + if (rest.size < 2) return Json.error("bad_args", "dm send <recipient> <text>") + val text = rest[1] + val ctx = Context.open(dataDir) + try { + ctx.prepare() + val recipient = ctx.requireUserHex(rest[0]) + + val template = ChatMessageEvent.build(text, listOf(PTag(recipient))) + val result = NIP17Factory().createMessageNIP17(template, ctx.signer) + + val recipientsOut = mutableListOf<Map<String, Any?>>() + for (wrap in result.wraps) { + val target = wrap.recipientPubKey() ?: continue + val relays = resolveDmRelays(ctx, target) + if (relays.isEmpty()) { + // A DM needs at least one relay that either the recipient + // or we agree on. If we can't even fall back to bootstrap + // we can't say "sent" honestly — surface it as an error. + return Json.error("no_dm_relays", target) + } + val ack = ctx.publish(wrap, relays) + recipientsOut.add( + mapOf( + "pubkey" to target, + "wrap_id" to wrap.id, + "published_to" to ack.filterValues { it }.keys.map { it.url }, + "relays_tried" to relays.map { it.url }, + ), + ) + } + + Json.writeLine( + mapOf( + "event_id" to result.msg.id, + "kind" to ChatMessageEvent.KIND, + "recipients" to recipientsOut, + ), + ) + return 0 + } finally { + ctx.close() + } + } + + private suspend fun list( + dataDir: DataDir, + rest: Array<String>, + ): Int { + val args = Args(rest) + val peerInput = args.flag("peer") + val sinceFlag = args.flags["since"]?.toLongOrNull() + val limit = args.intFlag("limit", Int.MAX_VALUE) + val timeoutSecs = args.longFlag("timeout", 8) + + val ctx = Context.open(dataDir) + try { + ctx.prepare() + val peerHex = peerInput?.let { ctx.requireUserHex(it) } + + val inbox = + ctx + .inboxRelays() + .ifEmpty { ctx.outboxRelays() } + .ifEmpty { ctx.bootstrapRelays() } + if (inbox.isEmpty()) return Json.error("no_inbox_relays", "configure relays or bootstrap defaults first") + + // Stateless queries (either --since or --peer provided) don't + // touch the cursor — the caller is asking a specific question. + // A no-flag invocation is the "advance my cursor" path, matching + // the Marmot syncIncoming convention. + val advanceCursor = peerInput == null && sinceFlag == null + val since = sinceFlag ?: ctx.state.giftWrapSince + + val filters = + inbox + .flatMap { filterGiftWrapsToPubkey(it, ctx.signer.pubKey, since) } + .groupBy { it.relay } + .mapValues { (_, v) -> v.map { it.filter } } + + val raw = ctx.drain(filters, timeoutMs = timeoutSecs * 1000) + + val messages = decryptChatMessages(ctx, raw, peerHex) + val out = + messages + .sortedBy { it.createdAt } + .takeLast(limit) + .map { it.toJson() } + + if (advanceCursor) { + val maxSeen = messages.maxOfOrNull { it.createdAt } + if (maxSeen != null) ctx.state.giftWrapSince = maxSeen + } + + Json.writeLine( + mapOf( + "peer" to peerHex, + "messages" to out, + ), + ) + return 0 + } finally { + ctx.close() + } + } + + private suspend fun await( + dataDir: DataDir, + rest: Array<String>, + ): Int { + val args = Args(rest) + val peerInput = args.requireFlag("peer") + val match = args.requireFlag("match") + val timeoutSecs = args.longFlag("timeout", 30) + + val ctx = Context.open(dataDir) + try { + ctx.prepare() + val peerHex = ctx.requireUserHex(peerInput) + + val inbox = + ctx + .inboxRelays() + .ifEmpty { ctx.outboxRelays() } + .ifEmpty { ctx.bootstrapRelays() } + if (inbox.isEmpty()) return Json.error("no_inbox_relays", "configure relays or bootstrap defaults first") + + val deadline = System.currentTimeMillis() + timeoutSecs * 1000 + // Remember how far we've already scanned on this invocation so + // subsequent polls only pull newly-arrived wraps. The 2-day + // lookback inside filterGiftWrapsToPubkey takes care of NIP-59's + // randomised created_at. + var since = ctx.state.giftWrapSince + + while (System.currentTimeMillis() < deadline) { + val filters = + inbox + .flatMap { filterGiftWrapsToPubkey(it, ctx.signer.pubKey, since) } + .groupBy { it.relay } + .mapValues { (_, v) -> v.map { it.filter } } + + val raw = ctx.drain(filters, timeoutMs = 3_000) + val messages = decryptChatMessages(ctx, raw, peerHex) + val hit = messages.firstOrNull { it.content.contains(match) } + if (hit != null) { + Json.writeLine(hit.toJson()) + return 0 + } + val maxSeen = messages.maxOfOrNull { it.createdAt } + if (maxSeen != null && (since == null || maxSeen > since)) since = maxSeen + delay(2_000) + } + throw AwaitTimeout("no DM from $peerHex matching '$match' within ${timeoutSecs}s") + } finally { + ctx.close() + } + } + + /** Where to deliver a gift-wrap addressed to [recipient]. */ + private suspend fun resolveDmRelays( + ctx: Context, + recipient: HexKey, + ): Set<NormalizedRelayUrl> { + val seed = ctx.bootstrapRelays() + val lists = RecipientRelayFetcher.fetchRelayLists(ctx.client, recipient, seed) + // 10050 inbox → 10002 read fallback → bootstrap as final safety net. + return lists.dmInboxOrFallback().toSet().ifEmpty { seed } + } + + private data class DecryptedDm( + val id: HexKey, + val wrapId: HexKey, + val from: HexKey, + val to: List<HexKey>, + val content: String, + val createdAt: Long, + val relay: String, + ) { + fun toJson(): Map<String, Any?> = + mapOf( + "id" to id, + "wrap_id" to wrapId, + "from" to from, + "to" to to, + "content" to content, + "created_at" to createdAt, + "relay" to relay, + ) + } + + private suspend fun decryptChatMessages( + ctx: Context, + raw: List<Pair<NormalizedRelayUrl, com.vitorpamplona.quartz.nip01Core.core.Event>>, + peerHex: HexKey?, + ): List<DecryptedDm> { + val seen = HashSet<HexKey>() + val out = mutableListOf<DecryptedDm>() + for ((relay, event) in raw) { + if (event !is GiftWrapEvent) continue + val sealed = event.unwrapOrNull(ctx.signer) as? SealedRumorEvent ?: continue + val inner = sealed.unsealOrNull(ctx.signer) as? ChatMessageEvent ?: continue + if (!seen.add(inner.id)) continue + val members = inner.groupMembers() + if (peerHex != null && peerHex !in members) continue + out.add( + DecryptedDm( + id = inner.id, + wrapId = event.id, + from = inner.pubKey, + to = inner.recipientsPubKey(), + content = inner.content, + createdAt = inner.createdAt, + relay = relay.url, + ), + ) + } + return out + } +} From 1e5b5d627624fad68267af86498a9b7d8ab3764f Mon Sep 17 00:00:00 2001 From: Claude <noreply@anthropic.com> Date: Thu, 23 Apr 2026 18:40:35 +0000 Subject: [PATCH 04/11] refactor(commons): share NIP-59 gift-wrap+seal unwrap helper MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three call sites did the same two-layer peel — kind:1059 → (nip44 decrypt) → kind:13 sealed rumor → (nip44 decrypt) → rumor Extracted as `GiftWrapEvent.unwrapAndUnsealOrNull(signer)` in commons/relayClient/nip17Dm/, and collapsed: - commons/marmot/MarmotIngest.kt — was an inline `when` switch; now a one-liner. Behaviour unchanged (still passes through if the inner isn't a seal, matching the previous defensive `else -> inner` branch). - desktopApp/Main.kt — dropped the nested unwrap/unseal block in the kind:1059 case of the LocalCache dispatcher. - cli/DmCommands.kt — replaced the manual chain in `decryptChatMessages`. Android's DecryptAndIndexProcessor keeps its two separate handlers (processNewGiftWrap / processNewSealedRumor) because the LocalCache pipeline re-enters on each peel — that's a different shape and not touched. No behaviour change on any platform. --- .../amethyst/cli/commands/DmCommands.kt | 5 +- .../amethyst/commons/marmot/MarmotIngest.kt | 21 +++------ .../relayClient/nip17Dm/GiftWrapDecryptor.kt | 46 +++++++++++++++++++ .../vitorpamplona/amethyst/desktop/Main.kt | 9 ++-- 4 files changed, 58 insertions(+), 23 deletions(-) create mode 100644 commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/relayClient/nip17Dm/GiftWrapDecryptor.kt diff --git a/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/DmCommands.kt b/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/DmCommands.kt index 287d9c29a..483d8b794 100644 --- a/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/DmCommands.kt +++ b/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/DmCommands.kt @@ -26,13 +26,13 @@ import com.vitorpamplona.amethyst.cli.Context import com.vitorpamplona.amethyst.cli.DataDir import com.vitorpamplona.amethyst.cli.Json import com.vitorpamplona.amethyst.commons.relayClient.nip17Dm.filterGiftWrapsToPubkey +import com.vitorpamplona.amethyst.commons.relayClient.nip17Dm.unwrapAndUnsealOrNull import com.vitorpamplona.quartz.marmot.RecipientRelayFetcher import com.vitorpamplona.quartz.nip01Core.core.HexKey import com.vitorpamplona.quartz.nip01Core.relay.normalizer.NormalizedRelayUrl import com.vitorpamplona.quartz.nip01Core.tags.people.PTag import com.vitorpamplona.quartz.nip17Dm.NIP17Factory import com.vitorpamplona.quartz.nip17Dm.messages.ChatMessageEvent -import com.vitorpamplona.quartz.nip59Giftwrap.seals.SealedRumorEvent import com.vitorpamplona.quartz.nip59Giftwrap.wraps.GiftWrapEvent import kotlinx.coroutines.delay @@ -258,8 +258,7 @@ object DmCommands { val out = mutableListOf<DecryptedDm>() for ((relay, event) in raw) { if (event !is GiftWrapEvent) continue - val sealed = event.unwrapOrNull(ctx.signer) as? SealedRumorEvent ?: continue - val inner = sealed.unsealOrNull(ctx.signer) as? ChatMessageEvent ?: continue + val inner = event.unwrapAndUnsealOrNull(ctx.signer) as? ChatMessageEvent ?: continue if (!seen.add(inner.id)) continue val members = inner.groupMembers() if (peerHex != null && peerHex !in members) continue diff --git a/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/marmot/MarmotIngest.kt b/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/marmot/MarmotIngest.kt index 870c75b52..7f5517e5c 100644 --- a/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/marmot/MarmotIngest.kt +++ b/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/marmot/MarmotIngest.kt @@ -20,6 +20,7 @@ */ package com.vitorpamplona.amethyst.commons.marmot +import com.vitorpamplona.amethyst.commons.relayClient.nip17Dm.unwrapAndUnsealOrNull import com.vitorpamplona.quartz.marmot.GroupEventResult import com.vitorpamplona.quartz.marmot.MarmotInboundProcessor import com.vitorpamplona.quartz.marmot.WelcomeResult @@ -27,7 +28,6 @@ import com.vitorpamplona.quartz.marmot.mip02Welcome.WelcomeEvent import com.vitorpamplona.quartz.marmot.mip03GroupMessages.GroupEvent import com.vitorpamplona.quartz.nip01Core.core.Event import com.vitorpamplona.quartz.nip01Core.core.HexKey -import com.vitorpamplona.quartz.nip59Giftwrap.seals.SealedRumorEvent import com.vitorpamplona.quartz.nip59Giftwrap.wraps.GiftWrapEvent /** @@ -96,19 +96,12 @@ suspend fun MarmotManager.ingest(event: Event): MarmotIngestResult = private suspend fun MarmotManager.ingestGiftWrap(wrap: GiftWrapEvent): MarmotIngestResult = try { - // NIP-59 wraps contain TWO encryption layers: - // kind:1059 gift wrap → kind:13 sealed rumor → the rumor itself. - // `GiftWrapEvent.unwrapOrNull` only peels the outer layer; when the - // result is a [SealedRumorEvent] we must unseal it to reach the - // kind:444 Welcome rumor. The old code checked `isWelcomeEvent` on - // the seal (kind:13) and always took the Ignored branch, which is - // why every inbound Welcome was silently dropped by the CLI and by - // any non-Amethyst consumer. - val rumor = - when (val inner = wrap.unwrapOrNull(signer) ?: return MarmotIngestResult.Ignored) { - is SealedRumorEvent -> inner.unsealOrNull(signer) ?: return MarmotIngestResult.Ignored - else -> inner - } + // NIP-59 wraps carry two encryption layers (kind:1059 → kind:13 → rumor). + // [unwrapAndUnsealOrNull] peels both so we land directly on the inner + // kind:444 Welcome rumor. Checking `isWelcomeEvent` on the seal itself + // (the old bug) always took the Ignored branch and silently dropped + // every inbound Welcome. + val rumor = wrap.unwrapAndUnsealOrNull(signer) ?: return MarmotIngestResult.Ignored if (!MarmotInboundProcessor.isWelcomeEvent(rumor) || rumor !is WelcomeEvent) { return MarmotIngestResult.Ignored } diff --git a/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/relayClient/nip17Dm/GiftWrapDecryptor.kt b/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/relayClient/nip17Dm/GiftWrapDecryptor.kt new file mode 100644 index 000000000..cf19417ca --- /dev/null +++ b/commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/relayClient/nip17Dm/GiftWrapDecryptor.kt @@ -0,0 +1,46 @@ +/* + * Copyright (c) 2025 Vitor Pamplona + * + * Permission is hereby granted, free of charge, to any person obtaining a copy of + * this software and associated documentation files (the "Software"), to deal in + * the Software without restriction, including without limitation the rights to use, + * copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the + * Software, and to permit persons to whom the Software is furnished to do so, + * subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +package com.vitorpamplona.amethyst.commons.relayClient.nip17Dm + +import com.vitorpamplona.quartz.nip01Core.core.Event +import com.vitorpamplona.quartz.nip01Core.signers.NostrSigner +import com.vitorpamplona.quartz.nip59Giftwrap.seals.SealedRumorEvent +import com.vitorpamplona.quartz.nip59Giftwrap.wraps.GiftWrapEvent + +/** + * Fully decrypt a NIP-59 gift wrap down to the inner rumor. + * + * A gift wrap carries two encryption layers: + * kind:1059 [GiftWrapEvent] → kind:13 [SealedRumorEvent] → the rumor. + * + * [GiftWrapEvent.unwrapOrNull] peels only the outer layer. Every non-Android + * consumer that looks at the rumor directly (the CLI's `dm list`, the desktop + * chat receive path, the Marmot welcome ingest in commons) needs both peels. + * + * If the inner content isn't a [SealedRumorEvent] (malformed, or a future + * NIP-59 variant that wraps a rumor directly), the outer unwrap result is + * returned as-is so callers can still route on kind. Either unwrap returning + * null propagates as null. + */ +suspend fun GiftWrapEvent.unwrapAndUnsealOrNull(signer: NostrSigner): Event? { + val inner = unwrapOrNull(signer) ?: return null + return if (inner is SealedRumorEvent) inner.unsealOrNull(signer) else inner +} diff --git a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/Main.kt b/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/Main.kt index fa46a6d2b..e34b57f39 100644 --- a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/Main.kt +++ b/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/Main.kt @@ -71,6 +71,7 @@ import androidx.compose.ui.window.Window import androidx.compose.ui.window.WindowPosition import androidx.compose.ui.window.application import androidx.compose.ui.window.rememberWindowState +import com.vitorpamplona.amethyst.commons.relayClient.nip17Dm.unwrapAndUnsealOrNull import com.vitorpamplona.amethyst.desktop.account.AccountManager import com.vitorpamplona.amethyst.desktop.account.AccountState import com.vitorpamplona.amethyst.desktop.cache.DesktopLocalCache @@ -1022,13 +1023,9 @@ fun MainContent( } is com.vitorpamplona.quartz.nip59Giftwrap.wraps.GiftWrapEvent -> { - // NIP-17: unwrap gift wrap → seal → inner event + // NIP-17: peel both the gift-wrap and sealed-rumor layers. scope.launch { - val seal = - event.unwrapOrNull(iAccount.signer) - as? com.vitorpamplona.quartz.nip59Giftwrap.seals.SealedRumorEvent - ?: return@launch - val innerEvent = seal.unsealOrNull(iAccount.signer) ?: return@launch + val innerEvent = event.unwrapAndUnsealOrNull(iAccount.signer) ?: return@launch when (innerEvent) { is com.vitorpamplona.quartz.nip17Dm.messages.ChatMessageEvent -> { val innerNote = localCache.getOrCreateNote(innerEvent.id) From 305ce20b521b0a78db4408a9ac3a16db5fc855bb Mon Sep 17 00:00:00 2001 From: Claude <noreply@anthropic.com> Date: Thu, 23 Apr 2026 19:34:52 +0000 Subject: [PATCH 05/11] feat(cli): NIP-17 file DMs (kind:15) on receive + send, strict 10050 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Receive: - decryptDms now accepts both ChatMessageEvent (kind:14) and ChatMessageEncryptedFileHeaderEvent (kind:15). The result is a sealed DecryptedDm hierarchy (TextDm / FileDm) with a `type: "text"|"file"` discriminator in JSON. File messages emit url, mime_type, encryption_algorithm, decryption_key, decryption_nonce, hash, original_hash, size, dim, blurhash. `dm await --match X` searches the text body for kind:14 and the URL for kind:15. Send: - New `dm send-file RECIPIENT URL --key HEX --nonce HEX [...]` verb. Builds a kind:15 ChatMessageEncryptedFileHeaderEvent via the existing ChatMessageEncryptedFileHeaderEvent.build + AESGCM(key, nonce), then publishes through NIP17Factory.createEncryptedFileNIP17. The file is expected to already be uploaded; carrying the upload itself in `amy` is the next change. Spec compliance (NIP-17 §6, "if no kind:10050, shouldn't try"): - resolveDmRelays now returns kind:10050 only by default. If the recipient has none, send/send-file exit with `no_dm_relays` and a message recommending `--allow-fallback`. The fallback chain (kind:10002 read marker → bootstrap) is preserved behind that opt-in flag for interop tests and brand-new accounts. Each recipient row in the send response now includes `relay_source` ("kind_10050" / "nip65_read" / "bootstrap") so callers can tell where the wraps actually went. Note: Quartz already enforces the NIP-17 §7 step-3 impersonation guard (seal pubkey is forced over the rumor's claimed pubkey inside `Rumor.mergeWith`), so no extra check is needed in the CLI. --- cli/README.md | 7 +- .../amethyst/cli/commands/DmCommands.kt | 343 ++++++++++++++---- 2 files changed, 275 insertions(+), 75 deletions(-) diff --git a/cli/README.md b/cli/README.md index 14450c09d..b585b492a 100644 --- a/cli/README.md +++ b/cli/README.md @@ -137,9 +137,10 @@ Run `amy --help` for the canonical list. As of today: | `marmot await message GID --match TEXT` | Block until a message containing `TEXT` lands. | | `marmot await rename GID --name NAME` | Block until GID's name matches. | | `marmot await epoch GID --min N` | Block until GID's MLS epoch is ≥ N. | -| `dm send RECIPIENT TEXT` | Send a NIP-17 gift-wrapped DM (kind:14 inside kind:1059). Publishes to the recipient's kind:10050 → kind:10002 read → our bootstrap pool. | -| `dm list [--peer NPUB] [--since TS] [--limit N] [--timeout SECS]` | Drain and decrypt gift wraps on our inbox relays. With neither `--peer` nor `--since` the gift-wrap cursor in `state.json` is advanced to the newest message seen. | -| `dm await --peer NPUB --match TEXT [--timeout SECS]` | Block until a DM from NPUB containing TEXT arrives. Timeout exits 124. | +| `dm send RECIPIENT TEXT [--allow-fallback]` | Send a NIP-17 gift-wrapped text DM (kind:14 inside kind:1059). Default delivers only to the recipient's kind:10050 (per NIP-17); pass `--allow-fallback` to fall back to kind:10002 read marker → bootstrap pool. | +| `dm send-file RECIPIENT URL --key HEX --nonce HEX [--mime-type M] [--hash H] [--original-hash H] [--size N] [--dim WxH] [--blurhash S] [--allow-fallback]` | Send a NIP-17 encrypted-file message (kind:15 inside kind:1059). The file must already be uploaded; `--key`/`--nonce` carry the AES-GCM material that recipients use to decrypt the bytes at `URL`. | +| `dm list [--peer NPUB] [--since TS] [--limit N] [--timeout SECS]` | Drain and decrypt gift wraps on our inbox relays. Returns kind:14 (text) and kind:15 (file) messages with a `type` discriminator. With neither `--peer` nor `--since` the gift-wrap cursor in `state.json` is advanced to the newest message seen. | +| `dm await --peer NPUB --match TEXT [--timeout SECS]` | Block until a DM from NPUB containing TEXT arrives (matches text content for kind:14, URL for kind:15). Timeout exits 124. | All `await` verbs accept `--timeout SECS` (default 30). Timeout exits 124 so scripts can distinguish "condition never happened" from "the command diff --git a/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/DmCommands.kt b/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/DmCommands.kt index 483d8b794..bf0d48326 100644 --- a/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/DmCommands.kt +++ b/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/DmCommands.kt @@ -28,28 +28,44 @@ import com.vitorpamplona.amethyst.cli.Json import com.vitorpamplona.amethyst.commons.relayClient.nip17Dm.filterGiftWrapsToPubkey import com.vitorpamplona.amethyst.commons.relayClient.nip17Dm.unwrapAndUnsealOrNull import com.vitorpamplona.quartz.marmot.RecipientRelayFetcher +import com.vitorpamplona.quartz.nip01Core.core.Event import com.vitorpamplona.quartz.nip01Core.core.HexKey +import com.vitorpamplona.quartz.nip01Core.core.hexToByteArray +import com.vitorpamplona.quartz.nip01Core.core.toHexKey import com.vitorpamplona.quartz.nip01Core.relay.normalizer.NormalizedRelayUrl import com.vitorpamplona.quartz.nip01Core.tags.people.PTag import com.vitorpamplona.quartz.nip17Dm.NIP17Factory +import com.vitorpamplona.quartz.nip17Dm.base.BaseDMGroupEvent +import com.vitorpamplona.quartz.nip17Dm.files.ChatMessageEncryptedFileHeaderEvent import com.vitorpamplona.quartz.nip17Dm.messages.ChatMessageEvent import com.vitorpamplona.quartz.nip59Giftwrap.wraps.GiftWrapEvent import kotlinx.coroutines.delay /** - * NIP-17 direct-message verbs. All three reuse the Quartz gift-wrap pipeline + * NIP-17 direct-message verbs. All verbs reuse the Quartz gift-wrap pipeline * (NIP-17 chat message → NIP-59 seal → NIP-59 gift wrap via NIP-44) — this - * command file is pure orchestration. + * file is pure orchestration over `quartz/` and `commons/`. + * + * Both kind:14 (text) and kind:15 (encrypted file header) are recognised on + * receive; `dm send-file` publishes kind:15 with the AES-GCM key + nonce + * carried in tags per NIP-17. + * + * NIP-17 says clients "shouldn't try" to send when the recipient hasn't + * published a kind:10050. By default we honour that: an empty kind:10050 + * is a hard error. `--allow-fallback` opts back into the kind:10002 read + * marker → bootstrap chain for the cases (interop tests, brand-new + * accounts) where strict mode is too strict. */ object DmCommands { suspend fun dispatch( dataDir: DataDir, tail: Array<String>, ): Int { - if (tail.isEmpty()) return Json.error("bad_args", "dm <send|list|await> …") + if (tail.isEmpty()) return Json.error("bad_args", "dm <send|send-file|list|await> …") val rest = tail.drop(1).toTypedArray() return when (tail[0]) { "send" -> send(dataDir, rest) + "send-file" -> sendFile(dataDir, rest) "list" -> list(dataDir, rest) "await" -> await(dataDir, rest) else -> Json.error("bad_args", "dm ${tail[0]}") @@ -60,50 +76,124 @@ object DmCommands { dataDir: DataDir, rest: Array<String>, ): Int { - if (rest.size < 2) return Json.error("bad_args", "dm send <recipient> <text>") + if (rest.size < 2) return Json.error("bad_args", "dm send <recipient> <text> [--allow-fallback]") val text = rest[1] + val args = Args(rest.drop(2).toTypedArray()) + val allowFallback = args.bool("allow-fallback") + val ctx = Context.open(dataDir) try { ctx.prepare() val recipient = ctx.requireUserHex(rest[0]) - val template = ChatMessageEvent.build(text, listOf(PTag(recipient))) val result = NIP17Factory().createMessageNIP17(template, ctx.signer) - - val recipientsOut = mutableListOf<Map<String, Any?>>() - for (wrap in result.wraps) { - val target = wrap.recipientPubKey() ?: continue - val relays = resolveDmRelays(ctx, target) - if (relays.isEmpty()) { - // A DM needs at least one relay that either the recipient - // or we agree on. If we can't even fall back to bootstrap - // we can't say "sent" honestly — surface it as an error. - return Json.error("no_dm_relays", target) - } - val ack = ctx.publish(wrap, relays) - recipientsOut.add( - mapOf( - "pubkey" to target, - "wrap_id" to wrap.id, - "published_to" to ack.filterValues { it }.keys.map { it.url }, - "relays_tried" to relays.map { it.url }, - ), - ) - } - - Json.writeLine( - mapOf( - "event_id" to result.msg.id, - "kind" to ChatMessageEvent.KIND, - "recipients" to recipientsOut, - ), - ) - return 0 + return publishWraps(ctx, result, allowFallback) } finally { ctx.close() } } + private suspend fun sendFile( + dataDir: DataDir, + rest: Array<String>, + ): Int { + if (rest.size < 2) { + return Json.error( + "bad_args", + "dm send-file <recipient> <url> --key <hex> --nonce <hex> [--mime-type <m>] [--hash <hex>] " + + "[--original-hash <hex>] [--size <n>] [--dim <WxH>] [--blurhash <s>] [--allow-fallback]", + ) + } + val url = rest[1] + val args = Args(rest.drop(2).toTypedArray()) + val keyHex = args.requireFlag("key") + val nonceHex = args.requireFlag("nonce") + val keyBytes = + runCatching { keyHex.hexToByteArray() }.getOrElse { + return Json.error("bad_args", "--key must be hex (got ${keyHex.length} chars)") + } + val nonceBytes = + runCatching { nonceHex.hexToByteArray() }.getOrElse { + return Json.error("bad_args", "--nonce must be hex (got ${nonceHex.length} chars)") + } + + val mimeType = args.flag("mime-type") + val hash = args.flag("hash") + val originalHash = args.flag("original-hash") + val size = args.flags["size"]?.toIntOrNull() + val blurhash = args.flag("blurhash") + val dimension = + args.flag("dim")?.let { raw -> + val match = + Regex("^(\\d+)x(\\d+)$").matchEntire(raw) + ?: return Json.error("bad_args", "--dim must be WxH (got '$raw')") + com.vitorpamplona.quartz.nip94FileMetadata.tags + .DimensionTag(match.groupValues[1].toInt(), match.groupValues[2].toInt()) + } + val allowFallback = args.bool("allow-fallback") + + val ctx = Context.open(dataDir) + try { + ctx.prepare() + val recipient = ctx.requireUserHex(rest[0]) + val cipher = + com.vitorpamplona.quartz.utils.ciphers + .AESGCM(keyBytes, nonceBytes) + val template = + ChatMessageEncryptedFileHeaderEvent.build( + to = listOf(PTag(recipient)), + url = url, + cipher = cipher, + mimeType = mimeType, + hash = hash, + size = size, + dimension = dimension, + blurhash = blurhash, + originalHash = originalHash, + ) + val result = NIP17Factory().createEncryptedFileNIP17(template, ctx.signer) + return publishWraps(ctx, result, allowFallback) + } finally { + ctx.close() + } + } + + private suspend fun publishWraps( + ctx: Context, + result: NIP17Factory.Result, + allowFallback: Boolean, + ): Int { + val recipientsOut = mutableListOf<Map<String, Any?>>() + for (wrap in result.wraps) { + val target = wrap.recipientPubKey() ?: continue + val resolution = resolveDmRelays(ctx, target, allowFallback) + if (resolution.relays.isEmpty()) { + return Json.error( + "no_dm_relays", + "$target has no kind:10050; pass --allow-fallback to use NIP-65 read or bootstrap", + ) + } + val ack = ctx.publish(wrap, resolution.relays) + recipientsOut.add( + mapOf( + "pubkey" to target, + "wrap_id" to wrap.id, + "published_to" to ack.filterValues { it }.keys.map { it.url }, + "relays_tried" to resolution.relays.map { it.url }, + "relay_source" to resolution.source, + ), + ) + } + Json.writeLine( + mapOf( + "event_id" to result.msg.id, + "kind" to result.msg.kind, + "recipients" to recipientsOut, + ), + ) + return 0 + } + private suspend fun list( dataDir: DataDir, rest: Array<String>, @@ -126,10 +216,6 @@ object DmCommands { .ifEmpty { ctx.bootstrapRelays() } if (inbox.isEmpty()) return Json.error("no_inbox_relays", "configure relays or bootstrap defaults first") - // Stateless queries (either --since or --peer provided) don't - // touch the cursor — the caller is asking a specific question. - // A no-flag invocation is the "advance my cursor" path, matching - // the Marmot syncIncoming convention. val advanceCursor = peerInput == null && sinceFlag == null val since = sinceFlag ?: ctx.state.giftWrapSince @@ -141,7 +227,7 @@ object DmCommands { val raw = ctx.drain(filters, timeoutMs = timeoutSecs * 1000) - val messages = decryptChatMessages(ctx, raw, peerHex) + val messages = decryptDms(ctx, raw, peerHex) val out = messages .sortedBy { it.createdAt } @@ -187,10 +273,6 @@ object DmCommands { if (inbox.isEmpty()) return Json.error("no_inbox_relays", "configure relays or bootstrap defaults first") val deadline = System.currentTimeMillis() + timeoutSecs * 1000 - // Remember how far we've already scanned on this invocation so - // subsequent polls only pull newly-arrived wraps. The 2-day - // lookback inside filterGiftWrapsToPubkey takes care of NIP-59's - // randomised created_at. var since = ctx.state.giftWrapSince while (System.currentTimeMillis() < deadline) { @@ -201,8 +283,11 @@ object DmCommands { .mapValues { (_, v) -> v.map { it.filter } } val raw = ctx.drain(filters, timeoutMs = 3_000) - val messages = decryptChatMessages(ctx, raw, peerHex) - val hit = messages.firstOrNull { it.content.contains(match) } + val messages = decryptDms(ctx, raw, peerHex) + // Match against the text body for kind:14 and against the URL + // for kind:15 — both are exposed as `searchText` so callers + // can grep for either with one --match flag. + val hit = messages.firstOrNull { match in it.searchText } if (hit != null) { Json.writeLine(hit.toJson()) return 0 @@ -217,28 +302,60 @@ object DmCommands { } } - /** Where to deliver a gift-wrap addressed to [recipient]. */ + /** + * Per NIP-17: kind:1059 should only be delivered to relays the recipient + * has advertised in their kind:10050. When that list is empty: + * - strict (default): refuse with no_dm_relays — caller must fix or + * explicitly opt into a fallback. + * - allowFallback=true: fall through to the NIP-65 read marker and then + * to our bootstrap pool. + */ private suspend fun resolveDmRelays( ctx: Context, recipient: HexKey, - ): Set<NormalizedRelayUrl> { + allowFallback: Boolean, + ): RelaySet { val seed = ctx.bootstrapRelays() val lists = RecipientRelayFetcher.fetchRelayLists(ctx.client, recipient, seed) - // 10050 inbox → 10002 read fallback → bootstrap as final safety net. - return lists.dmInboxOrFallback().toSet().ifEmpty { seed } + val dmInbox = lists.dmInbox.toSet() + if (dmInbox.isNotEmpty()) return RelaySet(dmInbox, "kind_10050") + if (!allowFallback) return RelaySet(emptySet(), "kind_10050") + val nip65Read = lists.nip65Read().toSet() + if (nip65Read.isNotEmpty()) return RelaySet(nip65Read, "nip65_read") + return RelaySet(seed, "bootstrap") } - private data class DecryptedDm( - val id: HexKey, - val wrapId: HexKey, - val from: HexKey, - val to: List<HexKey>, + private data class RelaySet( + val relays: Set<NormalizedRelayUrl>, + val source: String, + ) + + private sealed interface DecryptedDm { + val id: HexKey + val wrapId: HexKey + val from: HexKey + val to: List<HexKey> + val createdAt: Long + val relay: String + val searchText: String + + fun toJson(): Map<String, Any?> + } + + private data class TextDm( + override val id: HexKey, + override val wrapId: HexKey, + override val from: HexKey, + override val to: List<HexKey>, val content: String, - val createdAt: Long, - val relay: String, - ) { - fun toJson(): Map<String, Any?> = + override val createdAt: Long, + override val relay: String, + ) : DecryptedDm { + override val searchText: String get() = content + + override fun toJson(): Map<String, Any?> = mapOf( + "type" to "text", "id" to id, "wrap_id" to wrapId, "from" to from, @@ -249,31 +366,113 @@ object DmCommands { ) } - private suspend fun decryptChatMessages( + private data class FileDm( + override val id: HexKey, + override val wrapId: HexKey, + override val from: HexKey, + override val to: List<HexKey>, + val url: String, + val mimeType: String?, + val encryptionAlgo: String?, + val decryptionKey: String?, + val decryptionNonce: String?, + val hash: String?, + val originalHash: String?, + val size: Int?, + val dimensions: String?, + val blurhash: String?, + override val createdAt: Long, + override val relay: String, + ) : DecryptedDm { + override val searchText: String get() = url + + override fun toJson(): Map<String, Any?> { + val out = + mutableMapOf<String, Any?>( + "type" to "file", + "id" to id, + "wrap_id" to wrapId, + "from" to from, + "to" to to, + "url" to url, + "created_at" to createdAt, + "relay" to relay, + ) + if (mimeType != null) out["mime_type"] = mimeType + if (encryptionAlgo != null) out["encryption_algorithm"] = encryptionAlgo + if (decryptionKey != null) out["decryption_key"] = decryptionKey + if (decryptionNonce != null) out["decryption_nonce"] = decryptionNonce + if (hash != null) out["hash"] = hash + if (originalHash != null) out["original_hash"] = originalHash + if (size != null) out["size"] = size + if (dimensions != null) out["dim"] = dimensions + if (blurhash != null) out["blurhash"] = blurhash + return out + } + } + + private suspend fun decryptDms( ctx: Context, - raw: List<Pair<NormalizedRelayUrl, com.vitorpamplona.quartz.nip01Core.core.Event>>, + raw: List<Pair<NormalizedRelayUrl, Event>>, peerHex: HexKey?, ): List<DecryptedDm> { val seen = HashSet<HexKey>() val out = mutableListOf<DecryptedDm>() for ((relay, event) in raw) { if (event !is GiftWrapEvent) continue - val inner = event.unwrapAndUnsealOrNull(ctx.signer) as? ChatMessageEvent ?: continue + // Quartz's Rumor.mergeWith forces the inner event's pubkey to the + // seal author's, so the NIP-17 §7 step-3 impersonation check is + // already enforced upstream — no need to redo it here. + val inner = event.unwrapAndUnsealOrNull(ctx.signer) ?: continue + if (inner !is BaseDMGroupEvent) continue if (!seen.add(inner.id)) continue - val members = inner.groupMembers() - if (peerHex != null && peerHex !in members) continue - out.add( - DecryptedDm( + if (peerHex != null && peerHex !in inner.groupMembers()) continue + out.add(toDecrypted(inner, event.id, relay.url) ?: continue) + } + return out + } + + private fun toDecrypted( + inner: BaseDMGroupEvent, + wrapId: HexKey, + relayUrl: String, + ): DecryptedDm? = + when (inner) { + is ChatMessageEvent -> { + TextDm( id = inner.id, - wrapId = event.id, + wrapId = wrapId, from = inner.pubKey, to = inner.recipientsPubKey(), content = inner.content, createdAt = inner.createdAt, - relay = relay.url, - ), - ) + relay = relayUrl, + ) + } + + is ChatMessageEncryptedFileHeaderEvent -> { + FileDm( + id = inner.id, + wrapId = wrapId, + from = inner.pubKey, + to = inner.recipientsPubKey(), + url = inner.url(), + mimeType = inner.mimeType(), + encryptionAlgo = inner.algo(), + decryptionKey = inner.key()?.toHexKey(), + decryptionNonce = inner.nonce()?.toHexKey(), + hash = inner.hash(), + originalHash = inner.originalHash(), + size = inner.size(), + dimensions = inner.dimensions()?.let { "${it.width}x${it.height}" }, + blurhash = inner.blurhash(), + createdAt = inner.createdAt, + relay = relayUrl, + ) + } + + else -> { + null + } } - return out - } } From a7c1d45d93fcf0b3ebc606b86da9a41f021fa1bb Mon Sep 17 00:00:00 2001 From: Claude <noreply@anthropic.com> Date: Thu, 23 Apr 2026 19:47:41 +0000 Subject: [PATCH 06/11] refactor: promote desktop upload classes to commons/jvmMain MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Moves five JVM-only upload helpers from desktopApp into commons so the CLI can reuse the same Blossom upload + AES-GCM encryption path without depending on :desktopApp: desktop/service/upload/DesktopBlossomAuth.kt → commons/service/upload/BlossomAuth.kt desktop/service/upload/DesktopBlossomClient.kt → commons/service/upload/BlossomClient.kt desktop/service/upload/DesktopMediaCompressor.kt → commons/service/upload/MediaCompressor.kt desktop/service/upload/DesktopMediaMetadata.kt → commons/service/upload/MediaMetadata.kt desktop/service/upload/DesktopUploadOrchestrator.kt → commons/service/upload/UploadOrchestrator.kt API tweaks: - Drop the `Desktop` prefix on every class. - Rename the `DesktopMediaMetadata` object to `MediaMetadataReader` to avoid colliding with the `MediaMetadata` data class in the same package. - BlossomClient no longer reaches into desktop's `DesktopHttpClient`. Callers pass an `OkHttpClient` (or use the default one-shot client); desktop continues to inject its Tor-aware client. - Tighten BlossomClient's response handling — the OkHttp version on commons' classpath has a nullable `Response.body`, so explicitly null-check it. Mechanical updates to keep desktop building: - ComposeNoteDialog / ChatPane / DesktopUploadTracker switch to the new commons imports. - DimensionTag-from-MediaMetadata: cross-module smart cast no longer works on the public `width`/`height` properties — replace with `?.let { … }` chains. - commons/jvmMain now declares `commons-imaging` (used only by MediaCompressor's EXIF stripping). Tests come along too: file names follow the renamed classes (spotless ktlint enforces this). Next commit will add `amy dm send-file --file PATH --server URL`, which calls `UploadOrchestrator.uploadEncrypted(...)` directly. --- commons/build.gradle.kts | 3 ++ .../commons/service/upload/BlossomAuth.kt | 4 +- .../commons/service/upload/BlossomClient.kt | 22 +++++---- .../commons/service/upload/MediaCompressor.kt | 4 +- .../commons/service/upload/MediaMetadata.kt | 9 +++- .../service/upload/UploadOrchestrator.kt | 16 +++---- .../service/upload/DesktopUploadTracker.kt | 1 + .../amethyst/desktop/ui/ComposeNoteDialog.kt | 16 +++---- .../amethyst/desktop/ui/chats/ChatPane.kt | 10 ++-- ...ssomClientTest.kt => BlossomClientTest.kt} | 12 ++--- .../upload/DesktopUploadTrackerTest.kt | 2 + ...mpressorTest.kt => MediaCompressorTest.kt} | 14 +++--- ...dataTest.kt => MediaMetadataReaderTest.kt} | 46 +++++++++---------- ...ratorTest.kt => UploadOrchestratorTest.kt} | 24 +++++----- 14 files changed, 99 insertions(+), 84 deletions(-) rename desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopBlossomAuth.kt => commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/BlossomAuth.kt (95%) rename desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopBlossomClient.kt => commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/BlossomClient.kt (82%) rename desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopMediaCompressor.kt => commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/MediaCompressor.kt (95%) rename desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopMediaMetadata.kt => commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/MediaMetadata.kt (92%) rename desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopUploadOrchestrator.kt => commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/UploadOrchestrator.kt (90%) rename desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/{DesktopBlossomClientTest.kt => BlossomClientTest.kt} (95%) rename desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/{DesktopMediaCompressorTest.kt => MediaCompressorTest.kt} (90%) rename desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/{DesktopMediaMetadataTest.kt => MediaMetadataReaderTest.kt} (68%) rename desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/{DesktopUploadOrchestratorTest.kt => UploadOrchestratorTest.kt} (91%) diff --git a/commons/build.gradle.kts b/commons/build.gradle.kts index 56ec463bd..37b51fa5c 100644 --- a/commons/build.gradle.kts +++ b/commons/build.gradle.kts @@ -104,6 +104,9 @@ kotlin { // Secure key storage via OS keychain (macOS/Windows/Linux) implementation(libs.java.keyring) + + // EXIF stripping for image uploads (used by service/upload/MediaCompressor). + implementation(libs.commons.imaging) } } diff --git a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopBlossomAuth.kt b/commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/BlossomAuth.kt similarity index 95% rename from desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopBlossomAuth.kt rename to commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/BlossomAuth.kt index b93214497..9f02b737e 100644 --- a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopBlossomAuth.kt +++ b/commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/BlossomAuth.kt @@ -18,14 +18,14 @@ * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ -package com.vitorpamplona.amethyst.desktop.service.upload +package com.vitorpamplona.amethyst.commons.service.upload import com.vitorpamplona.quartz.nip01Core.core.HexKey import com.vitorpamplona.quartz.nip01Core.signers.NostrSigner import com.vitorpamplona.quartz.nipB7Blossom.BlossomAuthorizationEvent import java.util.Base64 -object DesktopBlossomAuth { +object BlossomAuth { suspend fun createUploadAuth( hash: HexKey, size: Long, diff --git a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopBlossomClient.kt b/commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/BlossomClient.kt similarity index 82% rename from desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopBlossomClient.kt rename to commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/BlossomClient.kt index 8bc6b99fe..2901a94f1 100644 --- a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopBlossomClient.kt +++ b/commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/BlossomClient.kt @@ -18,9 +18,8 @@ * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ -package com.vitorpamplona.amethyst.desktop.service.upload +package com.vitorpamplona.amethyst.commons.service.upload -import com.vitorpamplona.amethyst.desktop.network.DesktopHttpClient import com.vitorpamplona.quartz.nip01Core.core.JsonMapper import com.vitorpamplona.quartz.nipB7Blossom.BlossomUploadResult import kotlinx.coroutines.Dispatchers @@ -34,11 +33,16 @@ import okio.BufferedSink import okio.source import java.io.File -class DesktopBlossomClient( - private val clientOverride: OkHttpClient? = null, +/** + * Blossom HTTP client for JVM consumers (desktop + CLI). Owns no global + * state — pass a configured [OkHttpClient] (e.g. desktop's Tor-aware + * `DesktopHttpClient.currentClient()`) for proxying / connection pooling. + * The default constructor uses a fresh OkHttpClient — fine for one-shot + * uses such as the CLI. + */ +class BlossomClient( + private val okHttpClient: OkHttpClient = OkHttpClient(), ) { - private val okHttpClient: OkHttpClient get() = clientOverride ?: DesktopHttpClient.currentClient() - suspend fun upload( file: File, contentType: String, @@ -72,7 +76,8 @@ class DesktopBlossomClient( val reason = it.headers["X-Reason"] ?: it.code.toString() throw RuntimeException("Upload failed ($serverBaseUrl): $reason") } - JsonMapper.fromJson<BlossomUploadResult>(it.body.string()) + val body = it.body ?: throw RuntimeException("Upload to $serverBaseUrl returned no body") + JsonMapper.fromJson<BlossomUploadResult>(body.string()) } } @@ -103,7 +108,8 @@ class DesktopBlossomClient( val reason = it.headers["X-Reason"] ?: it.code.toString() throw RuntimeException("Upload failed ($serverBaseUrl): $reason") } - JsonMapper.fromJson<BlossomUploadResult>(it.body.string()) + val body = it.body ?: throw RuntimeException("Upload to $serverBaseUrl returned no body") + JsonMapper.fromJson<BlossomUploadResult>(body.string()) } } } diff --git a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopMediaCompressor.kt b/commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/MediaCompressor.kt similarity index 95% rename from desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopMediaCompressor.kt rename to commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/MediaCompressor.kt index c197441d7..bd4621eb4 100644 --- a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopMediaCompressor.kt +++ b/commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/MediaCompressor.kt @@ -18,14 +18,14 @@ * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ -package com.vitorpamplona.amethyst.desktop.service.upload +package com.vitorpamplona.amethyst.commons.service.upload import org.apache.commons.imaging.Imaging import org.apache.commons.imaging.formats.jpeg.exif.ExifRewriter import java.io.ByteArrayOutputStream import java.io.File -object DesktopMediaCompressor { +object MediaCompressor { fun stripExif(file: File): File { if (!file.name.lowercase().let { it.endsWith(".jpg") || it.endsWith(".jpeg") }) { return file diff --git a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopMediaMetadata.kt b/commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/MediaMetadata.kt similarity index 92% rename from desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopMediaMetadata.kt rename to commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/MediaMetadata.kt index 7981cb3a8..e1d60f046 100644 --- a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopMediaMetadata.kt +++ b/commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/MediaMetadata.kt @@ -18,7 +18,7 @@ * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ -package com.vitorpamplona.amethyst.desktop.service.upload +package com.vitorpamplona.amethyst.commons.service.upload import com.vitorpamplona.amethyst.commons.blurhash.toBlurhash import com.vitorpamplona.amethyst.commons.blurhash.toPlatformImage @@ -38,7 +38,12 @@ data class MediaMetadata( val thumbhash: String? = null, ) -object DesktopMediaMetadata { +/** + * Reads media metadata (sha256, size, mime-type, dimensions, blurhash, thumbhash) + * from a JVM [File]. Named to avoid colliding with the [MediaMetadata] data class + * in the same package. + */ +object MediaMetadataReader { fun compute(file: File): MediaMetadata { val bytes = file.readBytes() val hash = sha256(bytes).toHexKey() diff --git a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopUploadOrchestrator.kt b/commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/UploadOrchestrator.kt similarity index 90% rename from desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopUploadOrchestrator.kt rename to commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/UploadOrchestrator.kt index e22f00abc..54af97d0d 100644 --- a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopUploadOrchestrator.kt +++ b/commons/src/jvmMain/kotlin/com/vitorpamplona/amethyst/commons/service/upload/UploadOrchestrator.kt @@ -18,7 +18,7 @@ * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ -package com.vitorpamplona.amethyst.desktop.service.upload +package com.vitorpamplona.amethyst.commons.service.upload import com.vitorpamplona.quartz.nip01Core.core.toHexKey import com.vitorpamplona.quartz.nip01Core.signers.NostrSigner @@ -39,8 +39,8 @@ data class EncryptedUploadResult( val encryptedSize: Int, ) -class DesktopUploadOrchestrator( - private val client: DesktopBlossomClient = DesktopBlossomClient(), +class UploadOrchestrator( + private val client: BlossomClient = BlossomClient(), ) { suspend fun upload( file: File, @@ -52,17 +52,17 @@ class DesktopUploadOrchestrator( // 1. Strip EXIF if requested (JPEG only) val processedFile = if (stripExif) { - DesktopMediaCompressor.stripExif(file) + MediaCompressor.stripExif(file) } else { file } // 2. Compute metadata (hash, dimensions, blurhash) - val metadata = DesktopMediaMetadata.compute(processedFile) + val metadata = MediaMetadataReader.compute(processedFile) // 3. Create auth header val authHeader = - DesktopBlossomAuth.createUploadAuth( + BlossomAuth.createUploadAuth( hash = metadata.sha256, size = metadata.size, alt = alt ?: "Uploading ${file.name}", @@ -98,7 +98,7 @@ class DesktopUploadOrchestrator( signer: NostrSigner, ): EncryptedUploadResult { // 1. Compute pre-encryption metadata (dimensions, blurhash, mime, originalHash) - val metadata = DesktopMediaMetadata.compute(file) + val metadata = MediaMetadataReader.compute(file) // 2. Read file bytes and encrypt val plaintext = file.readBytes() @@ -110,7 +110,7 @@ class DesktopUploadOrchestrator( // 4. Create Blossom auth with encrypted hash and size val authHeader = - DesktopBlossomAuth.createUploadAuth( + BlossomAuth.createUploadAuth( hash = encryptedHash, size = encryptedSize.toLong(), alt = "Encrypted upload", diff --git a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopUploadTracker.kt b/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopUploadTracker.kt index e79244ecf..3350957ed 100644 --- a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopUploadTracker.kt +++ b/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopUploadTracker.kt @@ -20,6 +20,7 @@ */ package com.vitorpamplona.amethyst.desktop.service.upload +import com.vitorpamplona.amethyst.commons.service.upload.UploadResult import kotlinx.coroutines.flow.MutableStateFlow import kotlinx.coroutines.flow.asStateFlow diff --git a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/ui/ComposeNoteDialog.kt b/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/ui/ComposeNoteDialog.kt index bea6b8dd3..96a804554 100644 --- a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/ui/ComposeNoteDialog.kt +++ b/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/ui/ComposeNoteDialog.kt @@ -51,12 +51,12 @@ import androidx.compose.ui.draganddrop.DragAndDropEvent import androidx.compose.ui.draganddrop.DragAndDropTarget import androidx.compose.ui.unit.dp import androidx.compose.ui.window.Dialog +import com.vitorpamplona.amethyst.commons.service.upload.UploadOrchestrator +import com.vitorpamplona.amethyst.commons.service.upload.UploadResult import com.vitorpamplona.amethyst.desktop.DesktopPreferences import com.vitorpamplona.amethyst.desktop.account.AccountState import com.vitorpamplona.amethyst.desktop.network.DesktopRelayConnectionManager -import com.vitorpamplona.amethyst.desktop.service.upload.DesktopUploadOrchestrator import com.vitorpamplona.amethyst.desktop.service.upload.DesktopUploadTracker -import com.vitorpamplona.amethyst.desktop.service.upload.UploadResult import com.vitorpamplona.amethyst.desktop.ui.compose.ComposeRelayPicker import com.vitorpamplona.amethyst.desktop.ui.compose.RelayPickerState import com.vitorpamplona.amethyst.desktop.ui.media.ClipboardPasteHandler @@ -103,7 +103,7 @@ fun ComposeNoteDialog( val attachedFiles = remember { mutableStateListOf<File>() } val uploadTracker = remember { DesktopUploadTracker() } val uploadState by uploadTracker.state.collectAsState() - val orchestrator = remember { DesktopUploadOrchestrator() } + val orchestrator = remember { UploadOrchestrator() } var selectedServer by remember { mutableStateOf(DesktopPreferences.preferredBlossomServer) } var postAsPicture by remember { mutableStateOf(false) } @@ -482,11 +482,11 @@ private fun buildPictureMetas(results: List<UploadResult>): List<com.vitorpamplo mimeType = meta.mimeType, blurhash = meta.blurhash, dimension = - if (meta.width != null && meta.height != null) { - com.vitorpamplona.quartz.nip94FileMetadata.tags - .DimensionTag(meta.width, meta.height) - } else { - null + meta.width?.let { w -> + meta.height?.let { h -> + com.vitorpamplona.quartz.nip94FileMetadata.tags + .DimensionTag(w, h) + } }, hash = meta.sha256, size = meta.size.toInt(), diff --git a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/ui/chats/ChatPane.kt b/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/ui/chats/ChatPane.kt index 041626af1..07206b66a 100644 --- a/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/ui/chats/ChatPane.kt +++ b/desktopApp/src/jvmMain/kotlin/com/vitorpamplona/amethyst/desktop/ui/chats/ChatPane.kt @@ -86,6 +86,7 @@ import androidx.compose.ui.window.PopupProperties import com.vitorpamplona.amethyst.commons.model.IAccount import com.vitorpamplona.amethyst.commons.model.Note import com.vitorpamplona.amethyst.commons.model.cache.ICacheProvider +import com.vitorpamplona.amethyst.commons.service.upload.UploadOrchestrator import com.vitorpamplona.amethyst.commons.ui.chat.ChatMessageCompose import com.vitorpamplona.amethyst.commons.ui.chat.ChatroomHeader import com.vitorpamplona.amethyst.commons.ui.chat.DmBroadcastBanner @@ -97,7 +98,6 @@ import com.vitorpamplona.amethyst.commons.util.toTimeAgo import com.vitorpamplona.amethyst.commons.viewmodels.ChatNewMessageState import com.vitorpamplona.amethyst.commons.viewmodels.ChatroomFeedViewModel import com.vitorpamplona.amethyst.desktop.DesktopPreferences -import com.vitorpamplona.amethyst.desktop.service.upload.DesktopUploadOrchestrator import com.vitorpamplona.amethyst.desktop.ui.media.DesktopFilePicker import com.vitorpamplona.amethyst.desktop.ui.media.MediaAttachmentRow import com.vitorpamplona.quartz.nip01Core.hints.EventHintBundle @@ -801,7 +801,7 @@ private suspend fun sendEncryptedFiles( account: IAccount, cacheProvider: ICacheProvider, ) { - val orchestrator = DesktopUploadOrchestrator() + val orchestrator = UploadOrchestrator() val server = DesktopPreferences.preferredBlossomServer val recipients = roomKey.users.mapNotNull { cacheProvider.getUserIfExists(it) }.map { it.toPTag() } @@ -819,10 +819,8 @@ private suspend fun sendEncryptedFiles( hash = result.encryptedHash, size = result.encryptedSize, dimension = - if (result.metadata.width != null && result.metadata.height != null) { - DimensionTag(result.metadata.width, result.metadata.height) - } else { - null + result.metadata.width?.let { w -> + result.metadata.height?.let { h -> DimensionTag(w, h) } }, blurhash = result.metadata.blurhash, thumbhash = result.metadata.thumbhash, diff --git a/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopBlossomClientTest.kt b/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/BlossomClientTest.kt similarity index 95% rename from desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopBlossomClientTest.kt rename to desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/BlossomClientTest.kt index 75b94e6a8..514a8ee7f 100644 --- a/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopBlossomClientTest.kt +++ b/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/BlossomClientTest.kt @@ -37,7 +37,7 @@ import kotlin.test.assertEquals import kotlin.test.assertFailsWith import kotlin.test.assertTrue -class DesktopBlossomClientTest { +class BlossomClientTest { private fun mockOkHttp( responseCode: Int, body: String = "", @@ -67,7 +67,7 @@ class DesktopBlossomClientTest { runTest { val json = """{"url":"https://blossom.example.com/abc123.png","sha256":"abc123","size":1024}""" - val client = DesktopBlossomClient(mockOkHttp(200, json)) + val client = BlossomClient(mockOkHttp(200, json)) val file = File.createTempFile("test_", ".png") file.deleteOnExit() @@ -94,7 +94,7 @@ class DesktopBlossomClientTest { fun uploadFailureThrowsException() = runTest { val headers = Headers.headersOf("X-Reason", "File too large") - val client = DesktopBlossomClient(mockOkHttp(413, "", headers)) + val client = BlossomClient(mockOkHttp(413, "", headers)) val file = File.createTempFile("test_", ".png") file.deleteOnExit() @@ -119,7 +119,7 @@ class DesktopBlossomClientTest { @Test fun uploadFailureUsesStatusCodeWhenNoXReason() = runTest { - val client = DesktopBlossomClient(mockOkHttp(500)) + val client = BlossomClient(mockOkHttp(500)) val file = File.createTempFile("test_", ".png") file.deleteOnExit() @@ -159,7 +159,7 @@ class DesktopBlossomClientTest { .body("""{"url":"https://example.com/hash"}""".toResponseBody()) .build() - val client = DesktopBlossomClient(mockClient) + val client = BlossomClient(mockClient) val file = File.createTempFile("test_", ".png") file.deleteOnExit() file.writeBytes(byteArrayOf(1)) @@ -199,7 +199,7 @@ class DesktopBlossomClientTest { .body("""{"url":"https://example.com/hash"}""".toResponseBody()) .build() - val client = DesktopBlossomClient(mockClient) + val client = BlossomClient(mockClient) val file = File.createTempFile("test_", ".png") file.deleteOnExit() file.writeBytes(byteArrayOf(1)) diff --git a/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopUploadTrackerTest.kt b/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopUploadTrackerTest.kt index 193aa1019..ce8992c68 100644 --- a/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopUploadTrackerTest.kt +++ b/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopUploadTrackerTest.kt @@ -20,6 +20,8 @@ */ package com.vitorpamplona.amethyst.desktop.service.upload +import com.vitorpamplona.amethyst.commons.service.upload.MediaMetadata +import com.vitorpamplona.amethyst.commons.service.upload.UploadResult import com.vitorpamplona.quartz.nipB7Blossom.BlossomUploadResult import kotlinx.coroutines.flow.first import kotlinx.coroutines.test.runTest diff --git a/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopMediaCompressorTest.kt b/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/MediaCompressorTest.kt similarity index 90% rename from desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopMediaCompressorTest.kt rename to desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/MediaCompressorTest.kt index 6cef50f6f..52369ca95 100644 --- a/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopMediaCompressorTest.kt +++ b/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/MediaCompressorTest.kt @@ -27,7 +27,7 @@ import kotlin.test.Test import kotlin.test.assertEquals import kotlin.test.assertTrue -class DesktopMediaCompressorTest { +class MediaCompressorTest { @Test fun stripExifReturnsSameFileForPng() { val file = File.createTempFile("test_", ".png") @@ -35,7 +35,7 @@ class DesktopMediaCompressorTest { val img = BufferedImage(2, 2, BufferedImage.TYPE_INT_RGB) ImageIO.write(img, "png", file) - val result = DesktopMediaCompressor.stripExif(file) + val result = MediaCompressor.stripExif(file) // Should return the same file object since it's not JPEG assertEquals(file, result) @@ -48,7 +48,7 @@ class DesktopMediaCompressorTest { file.deleteOnExit() file.writeText("not a jpeg") - val result = DesktopMediaCompressor.stripExif(file) + val result = MediaCompressor.stripExif(file) assertEquals(file, result) file.delete() @@ -60,7 +60,7 @@ class DesktopMediaCompressorTest { file.deleteOnExit() file.writeBytes(byteArrayOf(0, 0, 0)) - val result = DesktopMediaCompressor.stripExif(file) + val result = MediaCompressor.stripExif(file) assertEquals(file, result) file.delete() @@ -71,7 +71,7 @@ class DesktopMediaCompressorTest { // Create a minimal JPEG without EXIF val file = createMinimalJpeg() try { - val result = DesktopMediaCompressor.stripExif(file) + val result = MediaCompressor.stripExif(file) // Should return the same file since there's no EXIF to strip assertEquals(file, result) } finally { @@ -87,7 +87,7 @@ class DesktopMediaCompressorTest { val img = BufferedImage(4, 4, BufferedImage.TYPE_INT_RGB) ImageIO.write(img, "jpg", file) - val result = DesktopMediaCompressor.stripExif(file) + val result = MediaCompressor.stripExif(file) // Result should be a valid file regardless assertTrue(result.exists()) @@ -107,7 +107,7 @@ class DesktopMediaCompressorTest { val img = BufferedImage(2, 2, BufferedImage.TYPE_INT_RGB) ImageIO.write(img, "jpg", file) - val result = DesktopMediaCompressor.stripExif(file) + val result = MediaCompressor.stripExif(file) assertTrue(result.exists()) if (result != file) result.delete() diff --git a/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopMediaMetadataTest.kt b/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/MediaMetadataReaderTest.kt similarity index 68% rename from desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopMediaMetadataTest.kt rename to desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/MediaMetadataReaderTest.kt index 2bb0f4016..53b2ba7a1 100644 --- a/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopMediaMetadataTest.kt +++ b/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/MediaMetadataReaderTest.kt @@ -29,60 +29,60 @@ import kotlin.test.assertNotNull import kotlin.test.assertNull import kotlin.test.assertTrue -class DesktopMediaMetadataTest { +class MediaMetadataReaderTest { // --- guessMimeType --- @Test fun guessMimeTypeForJpeg() { - assertEquals("image/jpeg", DesktopMediaMetadata.guessMimeType(File("photo.jpg"))) - assertEquals("image/jpeg", DesktopMediaMetadata.guessMimeType(File("photo.jpeg"))) - assertEquals("image/jpeg", DesktopMediaMetadata.guessMimeType(File("photo.JPEG"))) + assertEquals("image/jpeg", MediaMetadataReader.guessMimeType(File("photo.jpg"))) + assertEquals("image/jpeg", MediaMetadataReader.guessMimeType(File("photo.jpeg"))) + assertEquals("image/jpeg", MediaMetadataReader.guessMimeType(File("photo.JPEG"))) } @Test fun guessMimeTypeForPng() { - assertEquals("image/png", DesktopMediaMetadata.guessMimeType(File("image.png"))) + assertEquals("image/png", MediaMetadataReader.guessMimeType(File("image.png"))) } @Test fun guessMimeTypeForGif() { - assertEquals("image/gif", DesktopMediaMetadata.guessMimeType(File("anim.gif"))) + assertEquals("image/gif", MediaMetadataReader.guessMimeType(File("anim.gif"))) } @Test fun guessMimeTypeForWebp() { - assertEquals("image/webp", DesktopMediaMetadata.guessMimeType(File("image.webp"))) + assertEquals("image/webp", MediaMetadataReader.guessMimeType(File("image.webp"))) } @Test fun guessMimeTypeForSvg() { - assertEquals("image/svg+xml", DesktopMediaMetadata.guessMimeType(File("icon.svg"))) + assertEquals("image/svg+xml", MediaMetadataReader.guessMimeType(File("icon.svg"))) } @Test fun guessMimeTypeForAvif() { - assertEquals("image/avif", DesktopMediaMetadata.guessMimeType(File("photo.avif"))) + assertEquals("image/avif", MediaMetadataReader.guessMimeType(File("photo.avif"))) } @Test fun guessMimeTypeForVideoFormats() { - assertEquals("video/mp4", DesktopMediaMetadata.guessMimeType(File("clip.mp4"))) - assertEquals("video/webm", DesktopMediaMetadata.guessMimeType(File("clip.webm"))) - assertEquals("video/quicktime", DesktopMediaMetadata.guessMimeType(File("clip.mov"))) + assertEquals("video/mp4", MediaMetadataReader.guessMimeType(File("clip.mp4"))) + assertEquals("video/webm", MediaMetadataReader.guessMimeType(File("clip.webm"))) + assertEquals("video/quicktime", MediaMetadataReader.guessMimeType(File("clip.mov"))) } @Test fun guessMimeTypeForAudioFormats() { - assertEquals("audio/mpeg", DesktopMediaMetadata.guessMimeType(File("song.mp3"))) - assertEquals("audio/ogg", DesktopMediaMetadata.guessMimeType(File("track.ogg"))) - assertEquals("audio/wav", DesktopMediaMetadata.guessMimeType(File("sound.wav"))) - assertEquals("audio/flac", DesktopMediaMetadata.guessMimeType(File("lossless.flac"))) + assertEquals("audio/mpeg", MediaMetadataReader.guessMimeType(File("song.mp3"))) + assertEquals("audio/ogg", MediaMetadataReader.guessMimeType(File("track.ogg"))) + assertEquals("audio/wav", MediaMetadataReader.guessMimeType(File("sound.wav"))) + assertEquals("audio/flac", MediaMetadataReader.guessMimeType(File("lossless.flac"))) } @Test fun guessMimeTypeForUnknownExtension() { - assertEquals("application/octet-stream", DesktopMediaMetadata.guessMimeType(File("data.xyz"))) - assertEquals("application/octet-stream", DesktopMediaMetadata.guessMimeType(File("noext"))) + assertEquals("application/octet-stream", MediaMetadataReader.guessMimeType(File("data.xyz"))) + assertEquals("application/octet-stream", MediaMetadataReader.guessMimeType(File("noext"))) } // --- compute --- @@ -91,7 +91,7 @@ class DesktopMediaMetadataTest { fun computeForPngImage() { val file = createTempPng(width = 10, height = 5) try { - val meta = DesktopMediaMetadata.compute(file) + val meta = MediaMetadataReader.compute(file) assertEquals("image/png", meta.mimeType) assertTrue(meta.size > 0) @@ -110,7 +110,7 @@ class DesktopMediaMetadataTest { file.deleteOnExit() file.writeText("hello world") try { - val meta = DesktopMediaMetadata.compute(file) + val meta = MediaMetadataReader.compute(file) assertEquals("application/octet-stream", meta.mimeType) assertEquals(11L, meta.size) @@ -129,8 +129,8 @@ class DesktopMediaMetadataTest { file.deleteOnExit() file.writeBytes(byteArrayOf(1, 2, 3, 4, 5)) try { - val meta1 = DesktopMediaMetadata.compute(file) - val meta2 = DesktopMediaMetadata.compute(file) + val meta1 = MediaMetadataReader.compute(file) + val meta2 = MediaMetadataReader.compute(file) assertEquals(meta1.sha256, meta2.sha256) } finally { file.delete() @@ -143,7 +143,7 @@ class DesktopMediaMetadataTest { file.deleteOnExit() file.writeBytes(byteArrayOf(0, 0, 0)) try { - val meta = DesktopMediaMetadata.compute(file) + val meta = MediaMetadataReader.compute(file) assertEquals("video/mp4", meta.mimeType) assertNull(meta.width) assertNull(meta.height) diff --git a/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopUploadOrchestratorTest.kt b/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/UploadOrchestratorTest.kt similarity index 91% rename from desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopUploadOrchestratorTest.kt rename to desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/UploadOrchestratorTest.kt index b2ba3f3ba..dd0c9d3fe 100644 --- a/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/DesktopUploadOrchestratorTest.kt +++ b/desktopApp/src/jvmTest/kotlin/com/vitorpamplona/amethyst/desktop/service/upload/UploadOrchestratorTest.kt @@ -36,24 +36,24 @@ import kotlin.test.assertEquals import kotlin.test.assertNotNull import kotlin.test.assertTrue -class DesktopUploadOrchestratorTest { +class UploadOrchestratorTest { @BeforeTest fun setup() { - mockkObject(DesktopBlossomAuth) + mockkObject(BlossomAuth) coEvery { - DesktopBlossomAuth.createUploadAuth(any(), any(), any(), any()) + BlossomAuth.createUploadAuth(any(), any(), any(), any()) } returns "Nostr fakeAuthToken" } @AfterTest fun teardown() { - unmockkObject(DesktopBlossomAuth) + unmockkObject(BlossomAuth) } @Test fun uploadCallsClientWithCorrectParameters() = runTest { - val mockClient = mockk<DesktopBlossomClient>() + val mockClient = mockk<BlossomClient>() val fileSlot = slot<File>() val contentTypeSlot = slot<String>() val urlSlot = slot<String>() @@ -72,7 +72,7 @@ class DesktopUploadOrchestratorTest { size = 100, ) - val orchestrator = DesktopUploadOrchestrator(mockClient) + val orchestrator = UploadOrchestrator(mockClient) val file = File.createTempFile("test_", ".png") file.deleteOnExit() @@ -112,7 +112,7 @@ class DesktopUploadOrchestratorTest { @Test fun uploadPassesSameFileWhenNoStripExif() = runTest { - val mockClient = mockk<DesktopBlossomClient>() + val mockClient = mockk<BlossomClient>() val fileSlot = slot<File>() coEvery { @@ -124,7 +124,7 @@ class DesktopUploadOrchestratorTest { ) } returns BlossomUploadResult(url = "https://example.com/hash") - val orchestrator = DesktopUploadOrchestrator(mockClient) + val orchestrator = UploadOrchestrator(mockClient) val file = File.createTempFile("test_", ".txt") file.deleteOnExit() @@ -150,13 +150,13 @@ class DesktopUploadOrchestratorTest { @Test fun uploadComputesMetadata() = runTest { - val mockClient = mockk<DesktopBlossomClient>() + val mockClient = mockk<BlossomClient>() coEvery { mockClient.upload(any<java.io.File>(), any<String>(), any<String>(), any()) } returns BlossomUploadResult(url = "https://example.com/hash") - val orchestrator = DesktopUploadOrchestrator(mockClient) + val orchestrator = UploadOrchestrator(mockClient) val file = File.createTempFile("test_", ".txt") file.deleteOnExit() @@ -185,7 +185,7 @@ class DesktopUploadOrchestratorTest { @Test fun uploadPassesAuthHeaderToClient() = runTest { - val mockClient = mockk<DesktopBlossomClient>() + val mockClient = mockk<BlossomClient>() val authSlot = slot<String?>() coEvery { @@ -197,7 +197,7 @@ class DesktopUploadOrchestratorTest { ) } returns BlossomUploadResult(url = "https://example.com/hash") - val orchestrator = DesktopUploadOrchestrator(mockClient) + val orchestrator = UploadOrchestrator(mockClient) val file = File.createTempFile("test_", ".txt") file.deleteOnExit() From c614400b239f8d5c8318b33de4263d2725219390 Mon Sep 17 00:00:00 2001 From: Claude <noreply@anthropic.com> Date: Thu, 23 Apr 2026 19:51:06 +0000 Subject: [PATCH 07/11] feat(cli): amy dm send-file --file PATH (encrypt + upload + publish) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `dm send-file` now has two modes: - **Upload mode** — `dm send-file RECIPIENT --file PATH --server URL`: generate a fresh AES-GCM cipher, encrypt the file, upload the ciphertext to the Blossom server, then publish a kind:15 referencing the returned URL. Auto-detected metadata (encrypted hash + size, original sha256, mime type, image dimensions, blurhash) is folded into the kind:15 tags. The response surfaces the encryption key and nonce so the same encrypted blob can be re-shared without re-uploading. - **Reference mode** — the previous shape still works: `dm send-file RECIPIENT URL --key HEX --nonce HEX [...]`. Useful when the upload happened elsewhere (Android client, third-party tool) or for replaying a previous upload. Mode is selected by the presence of `--file`. Both paths share the same wrap-and-publish pipeline (`publishWraps`) and respect the strict NIP-17 §6 relay rule: kind:1059 only reaches the recipient's kind:10050 unless `--allow-fallback` is passed. Implementation entirely reuses the upload helpers just promoted to `commons/jvmMain/.../service/upload/`, so no new business logic lives in `cli/`. The Blossom auth event (kind:24242) is built and signed via `BlossomAuth.createUploadAuth(... ctx.signer ...)`. --- cli/README.md | 3 +- .../com/vitorpamplona/amethyst/cli/Main.kt | 10 +- .../amethyst/cli/commands/DmCommands.kt | 189 +++++++++++++----- 3 files changed, 154 insertions(+), 48 deletions(-) diff --git a/cli/README.md b/cli/README.md index b585b492a..a1077f37e 100644 --- a/cli/README.md +++ b/cli/README.md @@ -138,7 +138,8 @@ Run `amy --help` for the canonical list. As of today: | `marmot await rename GID --name NAME` | Block until GID's name matches. | | `marmot await epoch GID --min N` | Block until GID's MLS epoch is ≥ N. | | `dm send RECIPIENT TEXT [--allow-fallback]` | Send a NIP-17 gift-wrapped text DM (kind:14 inside kind:1059). Default delivers only to the recipient's kind:10050 (per NIP-17); pass `--allow-fallback` to fall back to kind:10002 read marker → bootstrap pool. | -| `dm send-file RECIPIENT URL --key HEX --nonce HEX [--mime-type M] [--hash H] [--original-hash H] [--size N] [--dim WxH] [--blurhash S] [--allow-fallback]` | Send a NIP-17 encrypted-file message (kind:15 inside kind:1059). The file must already be uploaded; `--key`/`--nonce` carry the AES-GCM material that recipients use to decrypt the bytes at `URL`. | +| `dm send-file RECIPIENT --file PATH --server URL [--mime-type M] [--allow-fallback]` | Encrypt the local file with a fresh AES-GCM cipher, upload the ciphertext to the Blossom server, then publish a kind:15 NIP-17 file message referencing the returned URL. The auto-detected hash, size, dimensions, and blurhash from the upload are folded into the event. The response also surfaces the encryption key + nonce so the same blob can be re-shared without re-uploading. | +| `dm send-file RECIPIENT URL --key HEX --nonce HEX [--mime-type M] [--hash H] [--original-hash H] [--size N] [--dim WxH] [--blurhash S] [--allow-fallback]` | Reference-mode variant: the file is already uploaded; `--key`/`--nonce` carry the AES-GCM material that recipients use to decrypt the bytes at `URL`. Useful when the upload happened elsewhere or to re-publish a previously-uploaded blob. | | `dm list [--peer NPUB] [--since TS] [--limit N] [--timeout SECS]` | Drain and decrypt gift wraps on our inbox relays. Returns kind:14 (text) and kind:15 (file) messages with a `type` discriminator. With neither `--peer` nor `--since` the gift-wrap cursor in `state.json` is advanced to the newest message seen. | | `dm await --peer NPUB --match TEXT [--timeout SECS]` | Block until a DM from NPUB containing TEXT arrives (matches text content for kind:14, URL for kind:15). Timeout exits 124. | diff --git a/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/Main.kt b/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/Main.kt index dd8d4dec4..1a2a8a127 100644 --- a/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/Main.kt +++ b/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/Main.kt @@ -195,8 +195,14 @@ private fun printUsage() { | |Direct messages (NIP-17): | dm send RECIPIENT TEXT send a gift-wrapped DM - | dm list [--peer NPUB] [--since TS] list decrypted DMs (advances cursor - | [--limit N] [--timeout SECS] only when no flags are passed) + | [--allow-fallback] (default: only deliver to recipient's kind:10050) + | dm send-file RECIPIENT --file PATH encrypt + upload to Blossom + publish kind:15 + | --server URL [--mime-type M] + | dm send-file RECIPIENT URL --key HEX reference-mode: file already uploaded + | --nonce HEX [--mime-type M] [--hash H] + | [--size N] [--dim WxH] [--blurhash S] + | dm list [--peer NPUB] [--since TS] list decrypted DMs (kind:14 text + + | [--limit N] [--timeout SECS] kind:15 file with `type` discriminator) | dm await --peer NPUB --match TEXT wait for a matching DM | [--timeout SECS] (default 30s, exit 124 on timeout) | diff --git a/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/DmCommands.kt b/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/DmCommands.kt index bf0d48326..d48652a86 100644 --- a/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/DmCommands.kt +++ b/cli/src/main/kotlin/com/vitorpamplona/amethyst/cli/commands/DmCommands.kt @@ -27,6 +27,7 @@ import com.vitorpamplona.amethyst.cli.DataDir import com.vitorpamplona.amethyst.cli.Json import com.vitorpamplona.amethyst.commons.relayClient.nip17Dm.filterGiftWrapsToPubkey import com.vitorpamplona.amethyst.commons.relayClient.nip17Dm.unwrapAndUnsealOrNull +import com.vitorpamplona.amethyst.commons.service.upload.UploadOrchestrator import com.vitorpamplona.quartz.marmot.RecipientRelayFetcher import com.vitorpamplona.quartz.nip01Core.core.Event import com.vitorpamplona.quartz.nip01Core.core.HexKey @@ -93,30 +94,129 @@ object DmCommands { } } + /** + * Two modes: + * + * - **Upload mode** (`--file PATH --server URL`): generate a random + * AES-GCM cipher, encrypt the local file, upload the ciphertext to + * the Blossom server, then publish a kind:15 referencing the + * returned URL. The auto-detected hash, size, dimensions, mime + * type, and blurhash from the upload are folded into the event. + * + * - **Reference mode** (positional URL + `--key HEX --nonce HEX`): + * the file is already uploaded somewhere; just publish a kind:15 + * pointing at it. Caller-supplied flags fill in the metadata. + * + * Mode selection: the `--file` flag turns on upload mode. Otherwise + * reference mode is required. + */ private suspend fun sendFile( dataDir: DataDir, rest: Array<String>, ): Int { - if (rest.size < 2) { - return Json.error( - "bad_args", - "dm send-file <recipient> <url> --key <hex> --nonce <hex> [--mime-type <m>] [--hash <hex>] " + - "[--original-hash <hex>] [--size <n>] [--dim <WxH>] [--blurhash <s>] [--allow-fallback]", - ) + if (rest.isEmpty()) return Json.error("bad_args", USAGE_SEND_FILE) + val args = Args(rest.drop(1).toTypedArray()) + val allowFallback = args.bool("allow-fallback") + val recipientInput = rest[0] + + val ctx = Context.open(dataDir) + try { + ctx.prepare() + val recipient = ctx.requireUserHex(recipientInput) + + val (template, summary) = + if (args.flag("file") != null) { + buildUploadModeTemplate(ctx, recipient, args) + ?: return 1 + } else { + buildReferenceModeTemplate(args, recipient) + ?: return 1 + } + + val result = NIP17Factory().createEncryptedFileNIP17(template, ctx.signer) + return publishWraps(ctx, result, allowFallback, extra = summary) + } finally { + ctx.close() } - val url = rest[1] - val args = Args(rest.drop(2).toTypedArray()) + } + + private suspend fun buildUploadModeTemplate( + ctx: Context, + recipient: com.vitorpamplona.quartz.nip01Core.core.HexKey, + args: Args, + ): Pair<com.vitorpamplona.quartz.nip01Core.signers.EventTemplate<ChatMessageEncryptedFileHeaderEvent>, Map<String, Any?>>? { + val file = java.io.File(args.requireFlag("file")) + if (!file.exists()) { + Json.error("bad_args", "file does not exist: ${file.absolutePath}") + return null + } + val server = args.requireFlag("server") + val cipher = + com.vitorpamplona.quartz.utils.ciphers + .AESGCM() + val orchestrator = UploadOrchestrator() + val uploaded = orchestrator.uploadEncrypted(file, cipher, server, ctx.signer) + val uploadedUrl = + uploaded.blossom.url ?: run { + Json.error("upload_failed", "Blossom server $server returned no URL") + return null + } + val mimeType = args.flag("mime-type") ?: uploaded.metadata.mimeType + val dimension = + uploaded.metadata.width?.let { w -> + uploaded.metadata.height?.let { h -> + com.vitorpamplona.quartz.nip94FileMetadata.tags + .DimensionTag(w, h) + } + } + val template = + ChatMessageEncryptedFileHeaderEvent.build( + to = listOf(PTag(recipient)), + url = uploadedUrl, + cipher = cipher, + mimeType = mimeType, + hash = uploaded.encryptedHash, + size = uploaded.encryptedSize, + dimension = dimension, + blurhash = uploaded.metadata.blurhash, + originalHash = uploaded.metadata.sha256, + ) + // Surface the cipher material on stdout so callers can re-share + // or republish the same encrypted blob without re-uploading. + val summary = + mapOf( + "url" to uploadedUrl, + "encryption_key" to cipher.keyBytes.toHexKey(), + "encryption_nonce" to cipher.nonce.toHexKey(), + "encrypted_hash" to uploaded.encryptedHash, + "encrypted_size" to uploaded.encryptedSize, + "original_hash" to uploaded.metadata.sha256, + "mime_type" to mimeType, + ) + return template to summary + } + + private fun buildReferenceModeTemplate( + args: Args, + recipient: com.vitorpamplona.quartz.nip01Core.core.HexKey, + ): Pair<com.vitorpamplona.quartz.nip01Core.signers.EventTemplate<ChatMessageEncryptedFileHeaderEvent>, Map<String, Any?>>? { + val url = + args.positionalOrNull(0) ?: run { + Json.error("bad_args", USAGE_SEND_FILE) + return null + } val keyHex = args.requireFlag("key") val nonceHex = args.requireFlag("nonce") val keyBytes = runCatching { keyHex.hexToByteArray() }.getOrElse { - return Json.error("bad_args", "--key must be hex (got ${keyHex.length} chars)") + Json.error("bad_args", "--key must be hex (got ${keyHex.length} chars)") + return null } val nonceBytes = runCatching { nonceHex.hexToByteArray() }.getOrElse { - return Json.error("bad_args", "--nonce must be hex (got ${nonceHex.length} chars)") + Json.error("bad_args", "--nonce must be hex (got ${nonceHex.length} chars)") + return null } - val mimeType = args.flag("mime-type") val hash = args.flag("hash") val originalHash = args.flag("original-hash") @@ -126,42 +226,40 @@ object DmCommands { args.flag("dim")?.let { raw -> val match = Regex("^(\\d+)x(\\d+)$").matchEntire(raw) - ?: return Json.error("bad_args", "--dim must be WxH (got '$raw')") + ?: run { + Json.error("bad_args", "--dim must be WxH (got '$raw')") + return null + } com.vitorpamplona.quartz.nip94FileMetadata.tags .DimensionTag(match.groupValues[1].toInt(), match.groupValues[2].toInt()) } - val allowFallback = args.bool("allow-fallback") - - val ctx = Context.open(dataDir) - try { - ctx.prepare() - val recipient = ctx.requireUserHex(rest[0]) - val cipher = - com.vitorpamplona.quartz.utils.ciphers - .AESGCM(keyBytes, nonceBytes) - val template = - ChatMessageEncryptedFileHeaderEvent.build( - to = listOf(PTag(recipient)), - url = url, - cipher = cipher, - mimeType = mimeType, - hash = hash, - size = size, - dimension = dimension, - blurhash = blurhash, - originalHash = originalHash, - ) - val result = NIP17Factory().createEncryptedFileNIP17(template, ctx.signer) - return publishWraps(ctx, result, allowFallback) - } finally { - ctx.close() - } + val cipher = + com.vitorpamplona.quartz.utils.ciphers + .AESGCM(keyBytes, nonceBytes) + val template = + ChatMessageEncryptedFileHeaderEvent.build( + to = listOf(PTag(recipient)), + url = url, + cipher = cipher, + mimeType = mimeType, + hash = hash, + size = size, + dimension = dimension, + blurhash = blurhash, + originalHash = originalHash, + ) + return template to emptyMap() } + private const val USAGE_SEND_FILE: String = + "dm send-file <recipient> [--file PATH --server URL | URL --key HEX --nonce HEX] " + + "[--mime-type M] [--hash HEX] [--original-hash HEX] [--size N] [--dim WxH] [--blurhash S] [--allow-fallback]" + private suspend fun publishWraps( ctx: Context, result: NIP17Factory.Result, allowFallback: Boolean, + extra: Map<String, Any?> = emptyMap(), ): Int { val recipientsOut = mutableListOf<Map<String, Any?>>() for (wrap in result.wraps) { @@ -184,13 +282,14 @@ object DmCommands { ), ) } - Json.writeLine( - mapOf( - "event_id" to result.msg.id, - "kind" to result.msg.kind, - "recipients" to recipientsOut, - ), - ) + val out = + buildMap { + put("event_id", result.msg.id) + put("kind", result.msg.kind) + putAll(extra) + put("recipients", recipientsOut) + } + Json.writeLine(out) return 0 } From 112801ba82151d6264cbed9b56bdb184914d8144 Mon Sep 17 00:00:00 2001 From: Claude <noreply@anthropic.com> Date: Thu, 23 Apr 2026 20:14:08 +0000 Subject: [PATCH 08/11] =?UTF-8?q?test(cli):=20amy=E2=86=94amy=20NIP-17=20D?= =?UTF-8?q?M=20interop=20harness?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds `tools/marmot-interop/dm-interop-headless.sh` — a zero-prompt end-to-end harness that runs two independent `amy` processes against a loopback nostr-rs-relay and verifies the NIP-17 DM surface. Six scenarios: dm-01 text round-trip (kind:14) in both directions dm-02 `dm list` surfaces prior exchange with `type:text` discriminator dm-03 strict kind:10050 refuses sends to an inboxless recipient (surfaces the `no_dm_relays` error JSON) dm-04 `--allow-fallback` opts into NIP-65 read / bootstrap chain (asserts a `relay_source: "bootstrap"` or `"nip65_read"` row) dm-05 file message reference mode (kind:15) — send-file with --key / --nonce, verify recipient decodes url + key + nonce + mime dm-06 no-flag `dm list` advances `state.giftWrapSince`; second call returns an empty `messages` array Shape matches the existing Marmot harness: reuses `lib.sh` for logging + result tracking, reuses `setup.sh` for the nostr-rs-relay lifecycle (`start_local_relay` / `stop_local_relay`), adds a slim `setup-dm.sh` preflight (amy + relay only, no whitenoise-rs / Marmot patches), and defines tests in `tests-dm.sh`. No new CI job yet — the relay build needs Rust + ~3 minutes on a cold cache, same constraint as the Marmot harness. Upload-mode (`dm send-file --file PATH --server URL`) is not scripted here: it needs a local Blossom server, which is out of scope for this pass. The shared upload classes are unit-tested on desktop at `desktopApp/src/jvmTest/kotlin/.../service/upload/`. --- cli/ROADMAP.md | 7 +- tools/marmot-interop/README.md | 31 ++- tools/marmot-interop/dm-interop-headless.sh | 98 +++++++++ tools/marmot-interop/headless/setup-dm.sh | 112 ++++++++++ tools/marmot-interop/headless/tests-dm.sh | 220 ++++++++++++++++++++ 5 files changed, 464 insertions(+), 4 deletions(-) create mode 100755 tools/marmot-interop/dm-interop-headless.sh create mode 100644 tools/marmot-interop/headless/setup-dm.sh create mode 100644 tools/marmot-interop/headless/tests-dm.sh diff --git a/cli/ROADMAP.md b/cli/ROADMAP.md index 466cd8410..61c227102 100644 --- a/cli/ROADMAP.md +++ b/cli/ROADMAP.md @@ -94,7 +94,12 @@ move anything, re-audit — you're probably duplicating logic. 7. **`amy zap send|verify`** (NIP-57). 8. **Distribution** — Homebrew + Scoop + `.deb` in the same release pipeline as desktop. Plan: `cli/plans/2026-04-21-cli-distribution.md`. -9. **Test suite** — end-to-end against a local relay. +9. **Test suite** — end-to-end against a local relay. Marmot interop is + covered by `tools/marmot-interop/marmot-interop-headless.sh`; NIP-17 + DM interop between two `amy` clients is covered by + `tools/marmot-interop/dm-interop-headless.sh` (text + file + strict + 10050 + fallback + cursor-advance). Neither runs in CI yet (both + need Rust + ~3 min cold relay build). 10. **Everything else in the matrix.** --- diff --git a/tools/marmot-interop/README.md b/tools/marmot-interop/README.md index f5fa97e0f..2fca06f1b 100644 --- a/tools/marmot-interop/README.md +++ b/tools/marmot-interop/README.md @@ -1,6 +1,6 @@ -# Marmot Interop Test Harness +# Interop Test Harnesses -Two flavours, same scenarios: +Two flavours of the Marmot harness, same scenarios: - **`marmot-interop.sh`** — interactive. Drives B/C via `wn` and **prompts the human** to perform each Amethyst-side step in the mobile UI (Identity A). @@ -10,7 +10,15 @@ Two flavours, same scenarios: end-to-end and exits with a pass/fail summary. Use this for CI and for iterating on the Nostr/Marmot plumbing without needing to touch a phone. -Both harnesses validate Amethyst against **whitenoise-rs** +A third, slimmer harness covers the NIP-17 DM surface: + +- **`dm-interop-headless.sh`** — two `amy` processes (Identity A and + Identity D) exchange NIP-17 DMs through the loopback nostr-rs-relay. + No whitenoise-rs required — only `amy` and the relay binary (which + is shared with the Marmot harness's checkout at + `state-headless/nostr-rs-relay/`). + +Both Marmot harnesses validate Amethyst against **whitenoise-rs** (https://github.com/marmot-protocol/whitenoise-rs), the reference Rust implementation that powers the White Noise Flutter app. Every test records a pass/fail/skip result into a tab-separated log, and the summary is printed at @@ -33,6 +41,23 @@ the end of the run. | 11 | Leave group | – | | 12 | Offline catch-up / replay | – | | 13 | KeyPackage rotation | – | + +### DM (amy ↔ amy, NIP-17) — `dm-interop-headless.sh` + +| # | Test | +|---|---| +| dm-01 | Text round-trip A↔D (kind:14) | +| dm-02 | `dm list` returns prior exchange with `type:text` discriminator | +| dm-03 | Strict kind:10050 refuses sends to an inboxless recipient | +| dm-04 | `--allow-fallback` opts into the NIP-65 read / bootstrap chain | +| dm-05 | File message reference mode round-trip (kind:15 with manual key/nonce) | +| dm-06 | No-flag `dm list` advances the gift-wrap cursor (second call empty) | + +**Note:** dm-05 validates the kind:15 wire format via reference mode +(caller supplies the URL + AES-GCM key/nonce). The upload-mode variant +(`dm send-file --file PATH --server URL`) needs a local Blossom server +and isn't scripted here — the upload classes are unit-tested on desktop +at `desktopApp/src/jvmTest/kotlin/.../service/upload/`. | 14 | Push notifications (MIP-05) | opt-in via `--transponder` | ## Prerequisites diff --git a/tools/marmot-interop/dm-interop-headless.sh b/tools/marmot-interop/dm-interop-headless.sh new file mode 100755 index 000000000..36414c256 --- /dev/null +++ b/tools/marmot-interop/dm-interop-headless.sh @@ -0,0 +1,98 @@ +#!/usr/bin/env bash +# +# dm-interop-headless.sh — zero-prompt NIP-17 DM interop harness. +# +# Two `amy` processes (Identity A and Identity D) talk to each other +# through a local nostr-rs-relay on ws://127.0.0.1:$RELAY_PORT. No +# whitenoise-rs, no Marmot, no public internet traffic. +# +# Usage: ./dm-interop-headless.sh [--port N] [--no-build] +# +set -uo pipefail + +SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" +REPO_ROOT="$(cd -- "$SCRIPT_DIR/../.." && pwd)" +STATE_DIR="$SCRIPT_DIR/state-dm-headless" +LOG_DIR="$STATE_DIR/logs" +A_DIR="$STATE_DIR/A" +D_DIR="$STATE_DIR/D" + +RUN_TS="$(date +%Y%m%d-%H%M%S)" +LOG_FILE="$LOG_DIR/run-$RUN_TS.log" +RESULTS_FILE="$STATE_DIR/results-$RUN_TS.tsv" + +AMY_BIN="$REPO_ROOT/cli/build/install/amy/bin/amy" + +# Share the nostr-rs-relay checkout with the Marmot harness to avoid +# rebuilding it twice. Override RELAY_REPO / RELAY_DATA if you want full +# isolation between runs. +RELAY_REPO="${RELAY_REPO:-$SCRIPT_DIR/state-headless/nostr-rs-relay}" +RELAY_BIN="$RELAY_REPO/target/release/nostr-rs-relay" +RELAY_DATA="$STATE_DIR/relay" +RELAY_PORT="${RELAY_PORT:-8090}" +RELAY_URL="ws://127.0.0.1:$RELAY_PORT" +NO_BUILD=0 + +A_NPUB="" +A_HEX="" +D_NPUB="" +D_HEX="" + +while [[ $# -gt 0 ]]; do + case "$1" in + --port) RELAY_PORT="$2"; RELAY_URL="ws://127.0.0.1:$RELAY_PORT"; shift ;; + --no-build) NO_BUILD=1 ;; + -h|--help) + sed -n '3,12p' "${BASH_SOURCE[0]}" | sed 's/^# \?//' + exit 0 ;; + *) printf 'unknown flag: %s\n' "$1" >&2; exit 2 ;; + esac + shift +done + +mkdir -p "$STATE_DIR" "$LOG_DIR" "$A_DIR" "$D_DIR" +: >"$LOG_FILE" +: >"$RESULTS_FILE" + +# Reuse the logging + result helpers from the Marmot harness. lib.sh +# also declares wn-specific helpers; they're harmless when unused. +# shellcheck source=lib.sh +source "$SCRIPT_DIR/lib.sh" + +# Reuse start_local_relay / stop_local_relay from setup.sh. preflight() +# there also builds whitenoise-rs, which we don't need — setup-dm.sh +# defines a slimmer preflight_dm(). +# shellcheck source=headless/setup.sh +source "$SCRIPT_DIR/headless/setup.sh" +# shellcheck source=headless/setup-dm.sh +source "$SCRIPT_DIR/headless/setup-dm.sh" +# shellcheck source=headless/helpers.sh +source "$SCRIPT_DIR/headless/helpers.sh" +# shellcheck source=headless/tests-dm.sh +source "$SCRIPT_DIR/headless/tests-dm.sh" + +cleanup() { + local rc=$? + trap - EXIT INT TERM HUP + stop_local_relay + print_summary + exit "$rc" +} +trap cleanup EXIT +trap 'exit 130' INT +trap 'exit 143' TERM +trap 'exit 129' HUP + +banner "Amethyst NIP-17 DM headless interop ($RUN_TS)" +preflight_dm +start_local_relay +ensure_identity_for A "$A_DIR" +ensure_identity_for D "$D_DIR" +configure_relays_dm + +test_01_dm_text_round_trip +test_02_dm_list_surfaces_history +test_03_dm_send_rejects_no_inbox +test_04_dm_send_allow_fallback +test_05_dm_file_reference_round_trip +test_06_dm_list_cursor_advance diff --git a/tools/marmot-interop/headless/setup-dm.sh b/tools/marmot-interop/headless/setup-dm.sh new file mode 100644 index 000000000..6bb7e764a --- /dev/null +++ b/tools/marmot-interop/headless/setup-dm.sh @@ -0,0 +1,112 @@ +# shellcheck shell=bash +# +# headless/setup-dm.sh — amy-only preflight + identity bootstrap for the +# NIP-17 DM interop harness. Much slimmer than the Marmot setup: +# +# - Builds `amy` (same retry-on-503 logic as setup.sh). +# - Builds nostr-rs-relay if missing. +# - Bootstraps two fresh amy identities (A and D), each with its own +# `--data-dir`, both pointed at the loopback relay. +# - Publishes kind:10050 (plus NIP-65) for both so NIP-17's strict +# recipient-inbox routing has something to resolve to. +# +# The heavy `start_local_relay` / `stop_local_relay` helpers live in +# headless/setup.sh and are sourced by the top-level harness. + +# --- preflight (amy + relay only, no wn / Marmot patches) ------------------- +preflight_dm() { + banner "Preflight (DM harness)" + for cmd in jq git cargo; do + if ! command -v "$cmd" >/dev/null 2>&1; then + fail_msg "missing required tool: $cmd" + exit 1 + fi + info "$cmd: $(command -v "$cmd")" + done + + if [[ ! -x "$AMY_BIN" ]]; then + if [[ "$NO_BUILD" -eq 1 ]]; then + fail_msg "amy not found at $AMY_BIN and --no-build set"; exit 1 + fi + # Gradle + jitpack/dl.google.com occasionally return transient 503s; + # retry so a single bad roll doesn't tank the whole harness. + local attempt max=4 + for attempt in $(seq 1 $max); do + step "building :cli:installDist (attempt $attempt/$max)" + if ( cd "$REPO_ROOT" && ./gradlew :cli:installDist ) 2>&1 | tee -a "$LOG_FILE" \ + && [[ -x "$AMY_BIN" ]]; then + break + fi + [[ "$attempt" -lt "$max" ]] && warn "gradle build failed — retrying" + done + fi + [[ -x "$AMY_BIN" ]] || { fail_msg "amy still missing after build"; exit 1; } + info "amy: $AMY_BIN" + + # nostr-rs-relay (same build path as the Marmot harness). + if [[ ! -x "$RELAY_BIN" ]]; then + if [[ "$NO_BUILD" -eq 1 ]]; then + fail_msg "nostr-rs-relay not found at $RELAY_BIN and --no-build set"; exit 1 + fi + if [[ ! -d "$RELAY_REPO/.git" ]]; then + step "cloning nostr-rs-relay into $RELAY_REPO" + git clone --depth 1 https://github.com/scsibug/nostr-rs-relay "$RELAY_REPO" \ + 2>&1 | tee -a "$LOG_FILE" + fi + local attempt max=4 + for attempt in $(seq 1 $max); do + step "building nostr-rs-relay (attempt $attempt/$max, ~3 min first run)" + ( cd "$RELAY_REPO" && cargo build --release --bin nostr-rs-relay ) \ + 2>&1 | tee -a "$LOG_FILE" + [[ -x "$RELAY_BIN" ]] && break + [[ "$attempt" -lt "$max" ]] && warn "nostr-rs-relay build failed — retrying" + done + [[ -x "$RELAY_BIN" ]] || { + fail_msg "nostr-rs-relay still missing after $max attempts"; exit 1 + } + fi + info "relay bin: $RELAY_BIN" +} + +# --- amy identity wrappers --------------------------------------------------- +# Two identities: A (sender) and D (recipient). We reuse A_DIR for parity +# with the existing harness files; D_DIR is new. +amy_a() { "$AMY_BIN" --data-dir "$A_DIR" "$@"; } +amy_d() { "$AMY_BIN" --data-dir "$D_DIR" "$@"; } + +# --- identity bootstrap ------------------------------------------------------ +ensure_identity_for() { + local who="$1" dir="$2" + step "initialising Identity $who (amy at $dir)" + local out + out=$("$AMY_BIN" --data-dir "$dir" init) || { + fail_msg "amy init failed for $who: $out"; exit 1 + } + local npub hex + npub=$(printf '%s' "$out" | jq -r '.npub') + hex=$(printf '%s' "$out" | jq -r '.hex') + case "$who" in + A) A_NPUB="$npub"; A_HEX="$hex" ;; + D) D_NPUB="$npub"; D_HEX="$hex" ;; + esac + info "$who npub: $npub" + info "$who hex: $hex" +} + +# --- relay wiring ------------------------------------------------------------ +# Point both identities at the loopback relay (all three buckets: +# nip65 / inbox / key_package), then publish each identity's kind:10050 +# so the DM strict-relay routing has something to resolve to. +configure_relays_dm() { + banner "Configuring relays → $RELAY_URL" + "$AMY_BIN" --data-dir "$A_DIR" relay add "$RELAY_URL" --type all >/dev/null + "$AMY_BIN" --data-dir "$D_DIR" relay add "$RELAY_URL" --type all >/dev/null + + step "publishing A's NIP-65 + kind:10050 lists" + amy_a relay publish-lists >>"$LOG_FILE" 2>&1 \ + || warn "amy_a relay publish-lists failed" + + step "publishing D's NIP-65 + kind:10050 lists" + amy_d relay publish-lists >>"$LOG_FILE" 2>&1 \ + || warn "amy_d relay publish-lists failed" +} diff --git a/tools/marmot-interop/headless/tests-dm.sh b/tools/marmot-interop/headless/tests-dm.sh new file mode 100644 index 000000000..a693ca675 --- /dev/null +++ b/tools/marmot-interop/headless/tests-dm.sh @@ -0,0 +1,220 @@ +# shellcheck shell=bash +# +# headless/tests-dm.sh — NIP-17 DM interop tests for two `amy` clients. +# +# Identity A (sender) and Identity D (recipient) each live in their own +# --data-dir and share one loopback nostr-rs-relay. Tests cover: +# +# dm-01 text round-trip (both directions) +# dm-02 dm list surfaces prior exchange with type:text discriminator +# dm-03 strict kind:10050 routing refuses sends to inboxless recipient +# dm-04 --allow-fallback opts into the NIP-65 read / bootstrap chain +# dm-05 file message reference mode round-trip (kind:15) +# dm-06 cursor advance (subsequent no-flag `dm list` is empty) + +# Wrap amy_json around either data-dir so the per-test code stays tight. +amy_json_for() { + local dir="$1"; shift + local out + if ! out=$("$AMY_BIN" --data-dir "$dir" "$@" 2>>"$LOG_FILE"); then + fail_msg "amy --data-dir $dir $*: exit $? (see $LOG_FILE)" + printf '%s\n' "$out" >>"$LOG_FILE" + return 1 + fi + printf '%s' "$out" +} + +amy_json_a() { amy_json_for "$A_DIR" "$@"; } +amy_json_d() { amy_json_for "$D_DIR" "$@"; } + +test_01_dm_text_round_trip() { + banner "DM-01 — text round-trip A↔D (kind:14)" + local id="dm-01 text round-trip" + + # A → D + local out_send + out_send=$(amy_json_a dm send "$D_NPUB" "hello from A") || { + record_result "$id" fail "A send failed"; return + } + local event_id recipients_ok + event_id=$(printf '%s' "$out_send" | jq -r '.event_id') + recipients_ok=$(printf '%s' "$out_send" \ + | jq -r '[.recipients[] | select(.relay_source == "kind_10050")] | length') + info "A sent kind:14 event_id=$event_id (to ${recipients_ok} kind:10050 inboxes)" + if [[ -z "$event_id" || "$event_id" == "null" ]]; then + record_result "$id" fail "A send response missing event_id"; return + fi + + # D awaits the text + if ! amy_json_d dm await --peer "$A_NPUB" --match "hello from A" --timeout 30 >/dev/null; then + record_result "$id" fail "D never received A's text"; return + fi + info "D received A's text" + + # D → A + amy_json_d dm send "$A_NPUB" "reply from D" >/dev/null || { + record_result "$id" fail "D send failed"; return + } + if ! amy_json_a dm await --peer "$D_NPUB" --match "reply from D" --timeout 30 >/dev/null; then + record_result "$id" fail "A never received D's reply"; return + fi + info "A received D's reply" + + record_result "$id" pass +} + +test_02_dm_list_surfaces_history() { + banner "DM-02 — dm list returns text messages with type discriminator" + local id="dm-02 dm list text history" + + local out + out=$(amy_json_a dm list --peer "$D_NPUB" --timeout 5) || { + record_result "$id" fail "amy dm list failed"; return + } + local text_count reply_present + text_count=$(printf '%s' "$out" \ + | jq -r '[.messages[] | select(.type == "text")] | length') + reply_present=$(printf '%s' "$out" \ + | jq -r '[.messages[] | select(.content == "reply from D")] | length') + info "A sees $text_count text message(s) with D; reply_present=$reply_present" + if [[ "$reply_present" == "1" ]]; then + record_result "$id" pass + else + record_result "$id" fail "D's reply missing from A's dm list" + fi +} + +test_03_dm_send_rejects_no_inbox() { + banner "DM-03 — strict kind:10050 refuses sends to inboxless recipient" + local id="dm-03 strict no_dm_relays" + + # Generate a throwaway identity but do NOT publish its kind:10050. + local tmpdir; tmpdir=$(mktemp -d "${STATE_DIR}/ghost.XXXXXX") + local ghost_out ghost_npub + ghost_out=$("$AMY_BIN" --data-dir "$tmpdir" init) || { + record_result "$id" fail "ghost init failed"; rm -rf "$tmpdir"; return + } + ghost_npub=$(printf '%s' "$ghost_out" | jq -r '.npub') + info "ghost: $ghost_npub (no relays advertised)" + + # A sends without --allow-fallback; amy should refuse. + local raw rc + raw=$("$AMY_BIN" --data-dir "$A_DIR" dm send "$ghost_npub" "should be rejected" 2>&1) + rc=$? + rm -rf "$tmpdir" + if [[ "$rc" -ne 0 ]] && printf '%s' "$raw" | grep -q '"error":"no_dm_relays"'; then + info "amy refused with no_dm_relays as expected (exit $rc)" + record_result "$id" pass + else + fail_msg "expected no_dm_relays error; got rc=$rc raw=$raw" + record_result "$id" fail "send to inboxless recipient did not refuse" + fi +} + +test_04_dm_send_allow_fallback() { + banner "DM-04 — --allow-fallback opts into NIP-65 read / bootstrap chain" + local id="dm-04 allow-fallback" + + # Same ghost pattern as dm-03. With --allow-fallback, resolution should + # fall through kind:10050 → NIP-65 read → bootstrap. Our bootstrap set + # always includes the loopback relay via the shared RelayConfig, so the + # publish should succeed even though the ghost has no 10050. + local tmpdir; tmpdir=$(mktemp -d "${STATE_DIR}/ghost.XXXXXX") + local ghost_out ghost_npub + ghost_out=$("$AMY_BIN" --data-dir "$tmpdir" init) || { + record_result "$id" fail "ghost init failed"; rm -rf "$tmpdir"; return + } + ghost_npub=$(printf '%s' "$ghost_out" | jq -r '.npub') + rm -rf "$tmpdir" + + local out source + out=$(amy_json_a dm send "$ghost_npub" "hi via fallback" --allow-fallback) || { + record_result "$id" fail "send with --allow-fallback failed"; return + } + # The ghost's own wrap will land in the recipients list; its source + # should be bootstrap (no 10050, no 10002) — which confirms the + # fallback actually fired. + source=$(printf '%s' "$out" \ + | jq -r --arg pk "${ghost_npub}" \ + '.recipients[] | select(.pubkey != $pk) | .relay_source' \ + | head -1) + info "relay_source for ghost wrap: $source" + if printf '%s' "$out" | jq -e '[.recipients[] | .relay_source] | index("bootstrap")' >/dev/null \ + || printf '%s' "$out" | jq -e '[.recipients[] | .relay_source] | index("nip65_read")' >/dev/null; then + record_result "$id" pass + else + record_result "$id" fail "no recipient reported a fallback relay_source" + fi +} + +test_05_dm_file_reference_round_trip() { + banner "DM-05 — file message reference mode (kind:15) round-trip" + local id="dm-05 file reference round-trip" + + # 64 hex chars = 32-byte AES-GCM key; 32 hex chars = 16-byte nonce. + local key="0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef" + local nonce="fedcba9876543210fedcba9876543210" + local url="https://example.test/blob/test-dm-05.bin" + local mime="application/octet-stream" + + amy_json_a dm send-file "$D_NPUB" "$url" \ + --key "$key" --nonce "$nonce" --mime-type "$mime" --size 1024 >/dev/null || { + record_result "$id" fail "A send-file failed"; return + } + info "A published kind:15 for $url" + + # D awaits — `dm await --match` searches URL for kind:15. + if ! amy_json_d dm await --peer "$A_NPUB" --match "$url" --timeout 30 >/dev/null; then + record_result "$id" fail "D never received the file message"; return + fi + + # Verify the recovered JSON carries all crypto material. + local raw + raw=$(amy_json_d dm list --peer "$A_NPUB" --timeout 5) || { + record_result "$id" fail "D dm list failed"; return + } + local fields + fields=$(printf '%s' "$raw" | jq -r \ + '[.messages[] | select(.type == "file" and .url == "'"$url"'")][0]') + if [[ -z "$fields" || "$fields" == "null" ]]; then + record_result "$id" fail "D did not decode a kind:15 matching the URL"; return + fi + + local got_key got_nonce got_mime + got_key=$(printf '%s' "$fields" | jq -r '.decryption_key') + got_nonce=$(printf '%s' "$fields" | jq -r '.decryption_nonce') + got_mime=$(printf '%s' "$fields" | jq -r '.mime_type') + info "D decoded key=${got_key:0:16}… nonce=${got_nonce:0:16}… mime=$got_mime" + assert_eq "$got_key" "$key" "$id" "decryption_key mismatch" || return + assert_eq "$got_nonce" "$nonce" "$id" "decryption_nonce mismatch" || return + assert_eq "$got_mime" "$mime" "$id" "mime_type mismatch" || return + + record_result "$id" pass +} + +test_06_dm_list_cursor_advance() { + banner "DM-06 — no-flag dm list advances the giftWrapSince cursor" + local id="dm-06 cursor advance" + + # First no-flag list: drains everything and advances the cursor to now. + local first_out first_count + first_out=$(amy_json_d dm list --timeout 5) || { + record_result "$id" fail "first dm list failed"; return + } + first_count=$(printf '%s' "$first_out" | jq -r '.messages | length') + info "first dm list returned $first_count message(s)" + + # Second no-flag list: nothing new should land. + local second_out second_count + second_out=$(amy_json_d dm list --timeout 5) || { + record_result "$id" fail "second dm list failed"; return + } + second_count=$(printf '%s' "$second_out" | jq -r '.messages | length') + info "second dm list returned $second_count message(s)" + + if [[ "$first_count" -ge 1 && "$second_count" -eq 0 ]]; then + record_result "$id" pass + else + record_result "$id" fail "expected first>=1, second==0; got first=$first_count second=$second_count" + fi +} From b0698e0a66a48e370c5e10f22dd3060a20158d74 Mon Sep 17 00:00:00 2001 From: Claude <noreply@anthropic.com> Date: Thu, 23 Apr 2026 20:37:22 +0000 Subject: [PATCH 09/11] test(cli): move tests/ out of tools/, separate marmot vs dm MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `tools/` is for libraries (arti-build); shell-based interop harnesses that drive the `amy` binary belong with the CLI module. New layout: cli/tests/ ├── lib.sh # shared logging + result tracking ├── headless/helpers.sh # shared amy_a / amy_json / assertions ├── marmot/ # MLS group-messaging interop (vs whitenoise-rs) │ ├── marmot-interop.sh │ ├── marmot-interop-headless.sh │ ├── setup.sh │ ├── tests-create.sh │ ├── tests-manage.sh │ ├── tests-extras.sh │ └── patches/ └── dm/ # NIP-17 DM interop (amy ↔ amy) ├── dm-interop-headless.sh ├── setup.sh └── tests-dm.sh Per-suite changes: - Each suite owns its own setup.sh; previously the Marmot setup was at headless/setup.sh and the DM setup at headless/setup-dm.sh, which implied shared infra they don't actually share. - `cli/tests/lib.sh` and `cli/tests/headless/helpers.sh` are the only shared bits across suites. - The DM harness still reuses Marmot's `start_local_relay` / `stop_local_relay` (sourced via `../marmot/setup.sh`) — same relay binary, no duplication. - All sourcing paths updated; REPO_ROOT now climbs three levels (was two when the harness lived under `tools/`); patches/ is now a sibling of marmot/setup.sh rather than under marmot/headless/. - `.gitignore` updated to cover both suites' state dirs. cli/ROADMAP.md and cli/tests/README.md updated to point at the new paths. No behavioural change — every test runs the same code; only the file layout and source paths moved. --- cli/ROADMAP.md | 10 ++-- cli/tests/.gitignore | 3 ++ {tools/marmot-interop => cli/tests}/README.md | 47 ++++++++++++++----- .../tests/dm}/dm-interop-headless.sh | 34 +++++++------- .../setup-dm.sh => cli/tests/dm/setup.sh | 4 +- .../headless => cli/tests/dm}/tests-dm.sh | 2 +- .../tests}/headless/helpers.sh | 2 +- {tools/marmot-interop => cli/tests}/lib.sh | 0 .../tests/marmot}/marmot-interop-headless.sh | 27 ++++++----- .../tests/marmot}/marmot-interop.sh | 5 +- .../patches/whitenoise-defaults-env.patch | 0 .../patches/whitenoise-discovery-env.patch | 0 .../patches/whitenoise-mock-keyring.patch | 0 .../whitenoise-skip-unprocessable-retry.patch | 0 .../headless => cli/tests/marmot}/setup.sh | 4 +- .../tests/marmot}/tests-create.sh | 2 +- .../tests/marmot}/tests-extras.sh | 2 +- .../tests/marmot}/tests-manage.sh | 2 +- tools/marmot-interop/.gitignore | 2 - 19 files changed, 87 insertions(+), 59 deletions(-) create mode 100644 cli/tests/.gitignore rename {tools/marmot-interop => cli/tests}/README.md (80%) rename {tools/marmot-interop => cli/tests/dm}/dm-interop-headless.sh (70%) rename tools/marmot-interop/headless/setup-dm.sh => cli/tests/dm/setup.sh (96%) rename {tools/marmot-interop/headless => cli/tests/dm}/tests-dm.sh (99%) rename {tools/marmot-interop => cli/tests}/headless/helpers.sh (95%) rename {tools/marmot-interop => cli/tests}/lib.sh (100%) rename {tools/marmot-interop => cli/tests/marmot}/marmot-interop-headless.sh (85%) rename {tools/marmot-interop => cli/tests/marmot}/marmot-interop.sh (99%) rename {tools/marmot-interop/headless => cli/tests/marmot}/patches/whitenoise-defaults-env.patch (100%) rename {tools/marmot-interop/headless => cli/tests/marmot}/patches/whitenoise-discovery-env.patch (100%) rename {tools/marmot-interop/headless => cli/tests/marmot}/patches/whitenoise-mock-keyring.patch (100%) rename {tools/marmot-interop/headless => cli/tests/marmot}/patches/whitenoise-skip-unprocessable-retry.patch (100%) rename {tools/marmot-interop/headless => cli/tests/marmot}/setup.sh (99%) rename {tools/marmot-interop/headless => cli/tests/marmot}/tests-create.sh (99%) rename {tools/marmot-interop/headless => cli/tests/marmot}/tests-extras.sh (99%) rename {tools/marmot-interop/headless => cli/tests/marmot}/tests-manage.sh (99%) delete mode 100644 tools/marmot-interop/.gitignore diff --git a/cli/ROADMAP.md b/cli/ROADMAP.md index 61c227102..8237e15c0 100644 --- a/cli/ROADMAP.md +++ b/cli/ROADMAP.md @@ -95,11 +95,11 @@ move anything, re-audit — you're probably duplicating logic. 8. **Distribution** — Homebrew + Scoop + `.deb` in the same release pipeline as desktop. Plan: `cli/plans/2026-04-21-cli-distribution.md`. 9. **Test suite** — end-to-end against a local relay. Marmot interop is - covered by `tools/marmot-interop/marmot-interop-headless.sh`; NIP-17 - DM interop between two `amy` clients is covered by - `tools/marmot-interop/dm-interop-headless.sh` (text + file + strict - 10050 + fallback + cursor-advance). Neither runs in CI yet (both - need Rust + ~3 min cold relay build). + covered by `cli/tests/marmot/marmot-interop-headless.sh`; NIP-17 DM + interop between two `amy` clients is covered by + `cli/tests/dm/dm-interop-headless.sh` (text + file + strict 10050 + + fallback + cursor-advance). Neither runs in CI yet (both need Rust + + ~3 min cold relay build). 10. **Everything else in the matrix.** --- diff --git a/cli/tests/.gitignore b/cli/tests/.gitignore new file mode 100644 index 000000000..f7bc17b06 --- /dev/null +++ b/cli/tests/.gitignore @@ -0,0 +1,3 @@ +marmot/state/ +marmot/state-headless/ +dm/state-dm-headless/ diff --git a/tools/marmot-interop/README.md b/cli/tests/README.md similarity index 80% rename from tools/marmot-interop/README.md rename to cli/tests/README.md index 2fca06f1b..f544930b9 100644 --- a/tools/marmot-interop/README.md +++ b/cli/tests/README.md @@ -1,22 +1,45 @@ -# Interop Test Harnesses +# amy CLI test harnesses -Two flavours of the Marmot harness, same scenarios: +Shell-based end-to-end harnesses that drive the `amy` CLI binary against a +loopback `nostr-rs-relay`. Layout: -- **`marmot-interop.sh`** — interactive. Drives B/C via `wn` and **prompts the - human** to perform each Amethyst-side step in the mobile UI (Identity A). - Use this for final UI verification. -- **`marmot-interop-headless.sh`** — zero prompts. Drives A via the `amy` CLI - (`./gradlew :cli:installDist`) and B/C via `wn`. Runs every scenario - end-to-end and exits with a pass/fail summary. Use this for CI and for - iterating on the Nostr/Marmot plumbing without needing to touch a phone. +``` +cli/tests/ +├── lib.sh # shared logging, results, assertions +├── headless/ # shared bits used by every harness +│ └── helpers.sh +├── marmot/ # Marmot / MLS group-messaging interop +│ ├── marmot-interop.sh # interactive — prompts Amethyst Android UI +│ ├── marmot-interop-headless.sh # zero-prompt +│ ├── setup.sh # preflight + wn + relay + identities +│ ├── tests-create.sh # tests 01–05 +│ ├── tests-manage.sh # tests 06–08, 11 +│ ├── tests-extras.sh # tests 09, 10, 12, 13 +│ └── patches/ # whitenoise-rs harness patches +└── dm/ # NIP-17 DM interop (amy ↔ amy) + ├── dm-interop-headless.sh + ├── setup.sh # preflight + identities + └── tests-dm.sh +``` + +The Marmot harnesses come in two flavours, same scenarios: + +- **`marmot/marmot-interop.sh`** — interactive. Drives B/C via `wn` and + **prompts the human** to perform each Amethyst-side step in the mobile UI + (Identity A). Use this for final UI verification. +- **`marmot/marmot-interop-headless.sh`** — zero prompts. Drives A via the + `amy` CLI (`./gradlew :cli:installDist`) and B/C via `wn`. Runs every + scenario end-to-end and exits with a pass/fail summary. Use this for CI + and for iterating on the Nostr/Marmot plumbing without needing to touch a + phone. A third, slimmer harness covers the NIP-17 DM surface: -- **`dm-interop-headless.sh`** — two `amy` processes (Identity A and +- **`dm/dm-interop-headless.sh`** — two `amy` processes (Identity A and Identity D) exchange NIP-17 DMs through the loopback nostr-rs-relay. No whitenoise-rs required — only `amy` and the relay binary (which is shared with the Marmot harness's checkout at - `state-headless/nostr-rs-relay/`). + `marmot/state-headless/nostr-rs-relay/`). Both Marmot harnesses validate Amethyst against **whitenoise-rs** (https://github.com/marmot-protocol/whitenoise-rs), the reference Rust @@ -42,7 +65,7 @@ the end of the run. | 12 | Offline catch-up / replay | – | | 13 | KeyPackage rotation | – | -### DM (amy ↔ amy, NIP-17) — `dm-interop-headless.sh` +### DM (amy ↔ amy, NIP-17) — `dm/dm-interop-headless.sh` | # | Test | |---|---| diff --git a/tools/marmot-interop/dm-interop-headless.sh b/cli/tests/dm/dm-interop-headless.sh similarity index 70% rename from tools/marmot-interop/dm-interop-headless.sh rename to cli/tests/dm/dm-interop-headless.sh index 36414c256..587f1b5fa 100755 --- a/tools/marmot-interop/dm-interop-headless.sh +++ b/cli/tests/dm/dm-interop-headless.sh @@ -11,7 +11,8 @@ set -uo pipefail SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" -REPO_ROOT="$(cd -- "$SCRIPT_DIR/../.." && pwd)" +REPO_ROOT="$(cd -- "$SCRIPT_DIR/../../.." && pwd)" +TESTS_DIR="$(cd -- "$SCRIPT_DIR/.." && pwd)" STATE_DIR="$SCRIPT_DIR/state-dm-headless" LOG_DIR="$STATE_DIR/logs" A_DIR="$STATE_DIR/A" @@ -26,7 +27,7 @@ AMY_BIN="$REPO_ROOT/cli/build/install/amy/bin/amy" # Share the nostr-rs-relay checkout with the Marmot harness to avoid # rebuilding it twice. Override RELAY_REPO / RELAY_DATA if you want full # isolation between runs. -RELAY_REPO="${RELAY_REPO:-$SCRIPT_DIR/state-headless/nostr-rs-relay}" +RELAY_REPO="${RELAY_REPO:-$TESTS_DIR/marmot/state-headless/nostr-rs-relay}" RELAY_BIN="$RELAY_REPO/target/release/nostr-rs-relay" RELAY_DATA="$STATE_DIR/relay" RELAY_PORT="${RELAY_PORT:-8090}" @@ -54,22 +55,23 @@ mkdir -p "$STATE_DIR" "$LOG_DIR" "$A_DIR" "$D_DIR" : >"$LOG_FILE" : >"$RESULTS_FILE" -# Reuse the logging + result helpers from the Marmot harness. lib.sh -# also declares wn-specific helpers; they're harmless when unused. -# shellcheck source=lib.sh -source "$SCRIPT_DIR/lib.sh" +# Reuse the logging + result helpers shared between every harness. +# lib.sh declares wn-specific helpers too — harmless when unused. +# shellcheck source=../lib.sh +source "$TESTS_DIR/lib.sh" -# Reuse start_local_relay / stop_local_relay from setup.sh. preflight() -# there also builds whitenoise-rs, which we don't need — setup-dm.sh +# Reuse start_local_relay / stop_local_relay from the Marmot harness's +# setup.sh — the relay lifecycle is identical. preflight() there also +# builds whitenoise-rs, which we don't need; setup.sh in this dir # defines a slimmer preflight_dm(). -# shellcheck source=headless/setup.sh -source "$SCRIPT_DIR/headless/setup.sh" -# shellcheck source=headless/setup-dm.sh -source "$SCRIPT_DIR/headless/setup-dm.sh" -# shellcheck source=headless/helpers.sh -source "$SCRIPT_DIR/headless/helpers.sh" -# shellcheck source=headless/tests-dm.sh -source "$SCRIPT_DIR/headless/tests-dm.sh" +# shellcheck source=../marmot/setup.sh +source "$TESTS_DIR/marmot/setup.sh" +# shellcheck source=setup.sh +source "$SCRIPT_DIR/setup.sh" +# shellcheck source=../headless/helpers.sh +source "$TESTS_DIR/headless/helpers.sh" +# shellcheck source=tests-dm.sh +source "$SCRIPT_DIR/tests-dm.sh" cleanup() { local rc=$? diff --git a/tools/marmot-interop/headless/setup-dm.sh b/cli/tests/dm/setup.sh similarity index 96% rename from tools/marmot-interop/headless/setup-dm.sh rename to cli/tests/dm/setup.sh index 6bb7e764a..148982375 100644 --- a/tools/marmot-interop/headless/setup-dm.sh +++ b/cli/tests/dm/setup.sh @@ -1,6 +1,6 @@ # shellcheck shell=bash # -# headless/setup-dm.sh — amy-only preflight + identity bootstrap for the +# setup.sh — amy-only preflight + identity bootstrap for the # NIP-17 DM interop harness. Much slimmer than the Marmot setup: # # - Builds `amy` (same retry-on-503 logic as setup.sh). @@ -11,7 +11,7 @@ # recipient-inbox routing has something to resolve to. # # The heavy `start_local_relay` / `stop_local_relay` helpers live in -# headless/setup.sh and are sourced by the top-level harness. +# the Marmot harness's setup.sh and are sourced by the top-level harness. # --- preflight (amy + relay only, no wn / Marmot patches) ------------------- preflight_dm() { diff --git a/tools/marmot-interop/headless/tests-dm.sh b/cli/tests/dm/tests-dm.sh similarity index 99% rename from tools/marmot-interop/headless/tests-dm.sh rename to cli/tests/dm/tests-dm.sh index a693ca675..945ee22d6 100644 --- a/tools/marmot-interop/headless/tests-dm.sh +++ b/cli/tests/dm/tests-dm.sh @@ -1,6 +1,6 @@ # shellcheck shell=bash # -# headless/tests-dm.sh — NIP-17 DM interop tests for two `amy` clients. +# tests-dm.sh — NIP-17 DM interop tests for two `amy` clients. # # Identity A (sender) and Identity D (recipient) each live in their own # --data-dir and share one loopback nostr-rs-relay. Tests cover: diff --git a/tools/marmot-interop/headless/helpers.sh b/cli/tests/headless/helpers.sh similarity index 95% rename from tools/marmot-interop/headless/helpers.sh rename to cli/tests/headless/helpers.sh index 7a7e10a65..7c9e55dc0 100644 --- a/tools/marmot-interop/headless/helpers.sh +++ b/cli/tests/headless/helpers.sh @@ -1,6 +1,6 @@ # shellcheck shell=bash # -# headless/helpers.sh — thin wrappers that keep the per-test code tight. +# helpers.sh — thin wrappers that keep the per-test code tight. # --- amy wrapper ------------------------------------------------------------- amy_a() { "$AMY_BIN" --data-dir "$A_DIR" "$@"; } diff --git a/tools/marmot-interop/lib.sh b/cli/tests/lib.sh similarity index 100% rename from tools/marmot-interop/lib.sh rename to cli/tests/lib.sh diff --git a/tools/marmot-interop/marmot-interop-headless.sh b/cli/tests/marmot/marmot-interop-headless.sh similarity index 85% rename from tools/marmot-interop/marmot-interop-headless.sh rename to cli/tests/marmot/marmot-interop-headless.sh index df2bd8308..c89ce806c 100755 --- a/tools/marmot-interop/marmot-interop-headless.sh +++ b/cli/tests/marmot/marmot-interop-headless.sh @@ -14,7 +14,8 @@ set -uo pipefail SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" -REPO_ROOT="$(cd -- "$SCRIPT_DIR/../.." && pwd)" +REPO_ROOT="$(cd -- "$SCRIPT_DIR/../../.." && pwd)" +TESTS_DIR="$(cd -- "$SCRIPT_DIR/.." && pwd)" STATE_DIR="$SCRIPT_DIR/state-headless" LOG_DIR="$STATE_DIR/logs" A_DIR="$STATE_DIR/A" @@ -66,19 +67,19 @@ mkdir -p "$STATE_DIR" "$LOG_DIR" "$A_DIR" "$B_DIR/logs" "$C_DIR/logs" : >"$RESULTS_FILE" # Reuse colours / logging / dump_daemon_diagnostics from the interactive harness. -# shellcheck source=lib.sh -source "$SCRIPT_DIR/lib.sh" +# shellcheck source=../lib.sh +source "$TESTS_DIR/lib.sh" -# shellcheck source=headless/setup.sh -source "$SCRIPT_DIR/headless/setup.sh" -# shellcheck source=headless/helpers.sh -source "$SCRIPT_DIR/headless/helpers.sh" -# shellcheck source=headless/tests-create.sh -source "$SCRIPT_DIR/headless/tests-create.sh" -# shellcheck source=headless/tests-manage.sh -source "$SCRIPT_DIR/headless/tests-manage.sh" -# shellcheck source=headless/tests-extras.sh -source "$SCRIPT_DIR/headless/tests-extras.sh" +# shellcheck source=setup.sh +source "$SCRIPT_DIR/setup.sh" +# shellcheck source=../headless/helpers.sh +source "$TESTS_DIR/headless/helpers.sh" +# shellcheck source=tests-create.sh +source "$SCRIPT_DIR/tests-create.sh" +# shellcheck source=tests-manage.sh +source "$SCRIPT_DIR/tests-manage.sh" +# shellcheck source=tests-extras.sh +source "$SCRIPT_DIR/tests-extras.sh" # Make sure Ctrl+C / SIGTERM / SIGHUP all run the full cleanup path — # otherwise wnd is nohup'd and keeps running after the script dies, diff --git a/tools/marmot-interop/marmot-interop.sh b/cli/tests/marmot/marmot-interop.sh similarity index 99% rename from tools/marmot-interop/marmot-interop.sh rename to cli/tests/marmot/marmot-interop.sh index f8f7e4702..bfca09ffa 100755 --- a/tools/marmot-interop/marmot-interop.sh +++ b/cli/tests/marmot/marmot-interop.sh @@ -10,6 +10,7 @@ set -uo pipefail SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" +TESTS_DIR="$(cd -- "$SCRIPT_DIR/.." && pwd)" STATE_DIR="$SCRIPT_DIR/state" LOG_DIR="$STATE_DIR/logs" B_DIR="$STATE_DIR/B" @@ -75,8 +76,8 @@ mkdir -p "$STATE_DIR" "$LOG_DIR" "$B_DIR/logs" "$C_DIR/logs" : >"$LOG_FILE" : >"$RESULTS_FILE" -# shellcheck source=lib.sh -source "$SCRIPT_DIR/lib.sh" +# shellcheck source=../lib.sh +source "$TESTS_DIR/lib.sh" # --- preflight --------------------------------------------------------------- preflight() { diff --git a/tools/marmot-interop/headless/patches/whitenoise-defaults-env.patch b/cli/tests/marmot/patches/whitenoise-defaults-env.patch similarity index 100% rename from tools/marmot-interop/headless/patches/whitenoise-defaults-env.patch rename to cli/tests/marmot/patches/whitenoise-defaults-env.patch diff --git a/tools/marmot-interop/headless/patches/whitenoise-discovery-env.patch b/cli/tests/marmot/patches/whitenoise-discovery-env.patch similarity index 100% rename from tools/marmot-interop/headless/patches/whitenoise-discovery-env.patch rename to cli/tests/marmot/patches/whitenoise-discovery-env.patch diff --git a/tools/marmot-interop/headless/patches/whitenoise-mock-keyring.patch b/cli/tests/marmot/patches/whitenoise-mock-keyring.patch similarity index 100% rename from tools/marmot-interop/headless/patches/whitenoise-mock-keyring.patch rename to cli/tests/marmot/patches/whitenoise-mock-keyring.patch diff --git a/tools/marmot-interop/headless/patches/whitenoise-skip-unprocessable-retry.patch b/cli/tests/marmot/patches/whitenoise-skip-unprocessable-retry.patch similarity index 100% rename from tools/marmot-interop/headless/patches/whitenoise-skip-unprocessable-retry.patch rename to cli/tests/marmot/patches/whitenoise-skip-unprocessable-retry.patch diff --git a/tools/marmot-interop/headless/setup.sh b/cli/tests/marmot/setup.sh similarity index 99% rename from tools/marmot-interop/headless/setup.sh rename to cli/tests/marmot/setup.sh index 66e3fff77..e475f9027 100644 --- a/tools/marmot-interop/headless/setup.sh +++ b/cli/tests/marmot/setup.sh @@ -1,6 +1,6 @@ # shellcheck shell=bash # -# headless/setup.sh — preflight + daemon lifecycle + identity bootstrap. +# setup.sh — preflight + daemon lifecycle + identity bootstrap. # Sourced from marmot-interop-headless.sh. # --- preflight --------------------------------------------------------------- @@ -90,7 +90,7 @@ preflight() { if [[ ! -f "$marker" ]]; then step "patching whitenoise-rs: $name" if ( cd "$WN_REPO" && patch -p1 --forward --reject-file=- \ - <"$SCRIPT_DIR/headless/patches/$name" >>"$LOG_FILE" 2>&1 ); then + <"$SCRIPT_DIR/patches/$name" >>"$LOG_FILE" 2>&1 ); then touch "$marker" # Invalidate the previous build so the patched source is picked up. rm -f "$WN_BIN" "$WND_BIN" diff --git a/tools/marmot-interop/headless/tests-create.sh b/cli/tests/marmot/tests-create.sh similarity index 99% rename from tools/marmot-interop/headless/tests-create.sh rename to cli/tests/marmot/tests-create.sh index 33c898c35..218851be0 100644 --- a/tools/marmot-interop/headless/tests-create.sh +++ b/cli/tests/marmot/tests-create.sh @@ -1,6 +1,6 @@ # shellcheck shell=bash # -# headless/tests-create.sh — tests 01..05. +# tests-create.sh — tests 01..05. # Focus: KeyPackage discovery, group creation, invites, initial messages. test_01_keypackage_discovery() { diff --git a/tools/marmot-interop/headless/tests-extras.sh b/cli/tests/marmot/tests-extras.sh similarity index 99% rename from tools/marmot-interop/headless/tests-extras.sh rename to cli/tests/marmot/tests-extras.sh index 6e8495856..b6b026a5a 100644 --- a/tools/marmot-interop/headless/tests-extras.sh +++ b/cli/tests/marmot/tests-extras.sh @@ -1,6 +1,6 @@ # shellcheck shell=bash # -# headless/tests-extras.sh — tests 09, 10, 12, 13. +# tests-extras.sh — tests 09, 10, 12, 13. # Focus: reactions/replies, concurrent commits, offline catchup, KP rotation. test_09_reply_react_unreact() { diff --git a/tools/marmot-interop/headless/tests-manage.sh b/cli/tests/marmot/tests-manage.sh similarity index 99% rename from tools/marmot-interop/headless/tests-manage.sh rename to cli/tests/marmot/tests-manage.sh index fd14c49f9..7923811ab 100644 --- a/tools/marmot-interop/headless/tests-manage.sh +++ b/cli/tests/marmot/tests-manage.sh @@ -1,6 +1,6 @@ # shellcheck shell=bash # -# headless/tests-manage.sh — tests 06, 07, 08, 11. +# tests-manage.sh — tests 06, 07, 08, 11. # Focus: removal, metadata rename, admin promote/demote, leave. test_06_member_removal() { diff --git a/tools/marmot-interop/.gitignore b/tools/marmot-interop/.gitignore deleted file mode 100644 index 1bcd48904..000000000 --- a/tools/marmot-interop/.gitignore +++ /dev/null @@ -1,2 +0,0 @@ -state/ -state-headless/ From b8a642b406836e0fb98fce90a7d5c8b339ea41a7 Mon Sep 17 00:00:00 2001 From: Claude <noreply@anthropic.com> Date: Thu, 23 Apr 2026 21:32:39 +0000 Subject: [PATCH 10/11] =?UTF-8?q?fix(cli/tests/dm):=20bind=20relay=20to=20?= =?UTF-8?q?127.0.0.2=20+=20dm-06=20=E2=86=92=20--since=20filter?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two fixes needed to make the DM harness actually pass end-to-end: 1. Bind the loopback relay to 127.0.0.2 instead of 127.0.0.1. Quartz's `RelayTag.parse` (called from `ChatMessageRelayListEvent. relays()`) rejects any URL for which `isLocalHost()` returns true — the substring check matches `"127.0.0.1"` among others. That means a kind:10050 event whose `relay` tag reads `ws://127.0.0.1:8090/` silently parses to an empty relay list, and `RecipientRelayFetcher` reports `dmInbox == []`. Under strict mode `dm send` then correctly refuses with `no_dm_relays` — hiding an otherwise-valid test setup. `127.0.0.2` is the cleanest fix: still pure loopback (no network traffic, no config needed), not matched by `isLocalHost()`, and Linux binds any IP in 127.0.0.0/8 without setup. The Marmot harness is unaffected because its tests never depend on parsing their own kind:10051/10002 relay tags back out — they hit the bootstrap fallback path. marmot/setup.sh's `start_local_relay` now reads `${RELAY_HOST:- 127.0.0.1}` so both harnesses can share it; the DM harness sets `RELAY_HOST=127.0.0.2` by default and exposes `--host` as an escape hatch. 2. Replace dm-06 cursor-advance check with `--since` filter check. The original assertion ("second no-flag `dm list` returns 0") was wrong for the actual design: `filterGiftWrapsToPubkey` always subtracts a 2-day lookback from `since` (required to catch NIP-59's randomised `created_at`), so a repeat no-flag query legitimately re-pulls everything in the last two days. The cursor advances the scan window for efficiency, not for idempotency. dm-06 now verifies `--since` actually filters: query with `--since <newest + 2 days + 1 hour>` — after the filter's lookback subtraction, the effective floor lands past every message, and the list comes back empty. Also: added `protoc` to the DM preflight checks so the missing-dep error comes before 4 failed cargo retries. Result on this machine: 6/6 pass, two clean runs in a row. --- cli/tests/README.md | 10 ++++++- cli/tests/dm/dm-interop-headless.sh | 12 ++++++-- cli/tests/dm/tests-dm.sh | 44 ++++++++++++++++++----------- cli/tests/marmot/setup.sh | 4 +-- 4 files changed, 47 insertions(+), 23 deletions(-) diff --git a/cli/tests/README.md b/cli/tests/README.md index f544930b9..f502b8c60 100644 --- a/cli/tests/README.md +++ b/cli/tests/README.md @@ -74,7 +74,15 @@ the end of the run. | dm-03 | Strict kind:10050 refuses sends to an inboxless recipient | | dm-04 | `--allow-fallback` opts into the NIP-65 read / bootstrap chain | | dm-05 | File message reference mode round-trip (kind:15 with manual key/nonce) | -| dm-06 | No-flag `dm list` advances the gift-wrap cursor (second call empty) | +| dm-06 | `dm list --since` filters out older messages (window-slide past the newest event returns 0) | + +**Relay binding note:** the DM harness binds the loopback relay to +`127.0.0.2` (not `127.0.0.1`) because Quartz's `RelayTag.parse` rejects +localhost URLs via `isLocalHost()` — so `ws://127.0.0.1` in a kind:10050 +event is silently stripped during recipient-relay resolution, which +would make strict-mode DM sends spuriously fail. `127.0.0.2` is still +pure loopback and isn't matched by that filter. Override with +`--host 127.0.0.5` etc. if `127.0.0.2` is taken. **Note:** dm-05 validates the kind:15 wire format via reference mode (caller supplies the URL + AES-GCM key/nonce). The upload-mode variant diff --git a/cli/tests/dm/dm-interop-headless.sh b/cli/tests/dm/dm-interop-headless.sh index 587f1b5fa..e916bc888 100755 --- a/cli/tests/dm/dm-interop-headless.sh +++ b/cli/tests/dm/dm-interop-headless.sh @@ -27,11 +27,16 @@ AMY_BIN="$REPO_ROOT/cli/build/install/amy/bin/amy" # Share the nostr-rs-relay checkout with the Marmot harness to avoid # rebuilding it twice. Override RELAY_REPO / RELAY_DATA if you want full # isolation between runs. +# Bind the loopback relay to 127.0.0.2 rather than 127.0.0.1 so Quartz's +# `isLocalHost()` filter doesn't silently strip it out of the kind:10050 +# inbox events during recipient-relay resolution. 127.0.0.2 is still pure +# loopback — no network traffic, no config needed. +RELAY_HOST="${RELAY_HOST:-127.0.0.2}" RELAY_REPO="${RELAY_REPO:-$TESTS_DIR/marmot/state-headless/nostr-rs-relay}" RELAY_BIN="$RELAY_REPO/target/release/nostr-rs-relay" RELAY_DATA="$STATE_DIR/relay" RELAY_PORT="${RELAY_PORT:-8090}" -RELAY_URL="ws://127.0.0.1:$RELAY_PORT" +RELAY_URL="ws://$RELAY_HOST:$RELAY_PORT" NO_BUILD=0 A_NPUB="" @@ -41,7 +46,8 @@ D_HEX="" while [[ $# -gt 0 ]]; do case "$1" in - --port) RELAY_PORT="$2"; RELAY_URL="ws://127.0.0.1:$RELAY_PORT"; shift ;; + --port) RELAY_PORT="$2"; RELAY_URL="ws://$RELAY_HOST:$RELAY_PORT"; shift ;; + --host) RELAY_HOST="$2"; RELAY_URL="ws://$RELAY_HOST:$RELAY_PORT"; shift ;; --no-build) NO_BUILD=1 ;; -h|--help) sed -n '3,12p' "${BASH_SOURCE[0]}" | sed 's/^# \?//' @@ -97,4 +103,4 @@ test_02_dm_list_surfaces_history test_03_dm_send_rejects_no_inbox test_04_dm_send_allow_fallback test_05_dm_file_reference_round_trip -test_06_dm_list_cursor_advance +test_06_dm_list_since_filter diff --git a/cli/tests/dm/tests-dm.sh b/cli/tests/dm/tests-dm.sh index 945ee22d6..51e1f448b 100644 --- a/cli/tests/dm/tests-dm.sh +++ b/cli/tests/dm/tests-dm.sh @@ -192,29 +192,39 @@ test_05_dm_file_reference_round_trip() { record_result "$id" pass } -test_06_dm_list_cursor_advance() { - banner "DM-06 — no-flag dm list advances the giftWrapSince cursor" - local id="dm-06 cursor advance" +test_06_dm_list_since_filter() { + banner "DM-06 — --since filters out older messages" + local id="dm-06 since filter" - # First no-flag list: drains everything and advances the cursor to now. - local first_out first_count - first_out=$(amy_json_d dm list --timeout 5) || { - record_result "$id" fail "first dm list failed"; return + # Baseline: a --peer query (stateless) should surface every message so + # far. We need at least one to have a meaningful window to slide past. + local baseline baseline_count + baseline=$(amy_json_d dm list --peer "$A_NPUB" --timeout 5) || { + record_result "$id" fail "baseline dm list failed"; return } - first_count=$(printf '%s' "$first_out" | jq -r '.messages | length') - info "first dm list returned $first_count message(s)" + baseline_count=$(printf '%s' "$baseline" | jq -r '.messages | length') + info "baseline messages with A: $baseline_count" + if [[ "$baseline_count" -lt 1 ]]; then + record_result "$id" fail "no messages to filter — earlier tests didn't populate"; return + fi - # Second no-flag list: nothing new should land. - local second_out second_count - second_out=$(amy_json_d dm list --timeout 5) || { - record_result "$id" fail "second dm list failed"; return + # Now query with --since set to a timestamp 2 days after the newest + # message. filterGiftWrapsToPubkey subtracts its own two-day lookback + # window, so the effective `since` lands right after the newest event + # — nothing should come back. + local newest + newest=$(printf '%s' "$baseline" | jq -r '[.messages[].created_at] | max') + local future=$(( newest + 2 * 24 * 60 * 60 + 3600 )) + local after after_count + after=$(amy_json_d dm list --peer "$A_NPUB" --since "$future" --timeout 5) || { + record_result "$id" fail "since-query failed"; return } - second_count=$(printf '%s' "$second_out" | jq -r '.messages | length') - info "second dm list returned $second_count message(s)" + after_count=$(printf '%s' "$after" | jq -r '.messages | length') + info "after --since $future: $after_count message(s)" - if [[ "$first_count" -ge 1 && "$second_count" -eq 0 ]]; then + if [[ "$after_count" -eq 0 ]]; then record_result "$id" pass else - record_result "$id" fail "expected first>=1, second==0; got first=$first_count second=$second_count" + record_result "$id" fail "expected 0 after future --since; got $after_count" fi } diff --git a/cli/tests/marmot/setup.sh b/cli/tests/marmot/setup.sh index e475f9027..61d8abbb3 100644 --- a/cli/tests/marmot/setup.sh +++ b/cli/tests/marmot/setup.sh @@ -170,7 +170,7 @@ description = "Loopback relay for marmot-interop-headless.sh — do not use for data_directory = "$RELAY_DATA" [network] -address = "127.0.0.1" +address = "${RELAY_HOST:-127.0.0.1}" port = $RELAY_PORT [options] @@ -198,7 +198,7 @@ EOF local deadline=$(( $(date +%s) + 20 )) while [[ $(date +%s) -lt $deadline ]]; do - if curl -sSf -m 1 "http://127.0.0.1:$RELAY_PORT/" >/dev/null 2>&1; then + if curl -sSf -m 1 "http://${RELAY_HOST:-127.0.0.1}:$RELAY_PORT/" >/dev/null 2>&1; then info "relay up" return 0 fi From bd88f68dcda40406d0cbbdbdd0d50c1c1cc3f81b Mon Sep 17 00:00:00 2001 From: Claude <noreply@anthropic.com> Date: Thu, 23 Apr 2026 21:36:52 +0000 Subject: [PATCH 11/11] docs: point README / DEVELOPMENT / amy-expert at cli/tests/ MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit After the harness move from tools/marmot-interop/ to cli/tests/, three doc locations were still describing the old layout: - cli/DEVELOPMENT.md § Testing — the "round-trip" row pointed at a non-existent `cli/src/test/resources/scripts/`, and the "interop with other clients" row claimed it was out of scope. Both now name `cli/tests/marmot/` and `cli/tests/dm/` with what each covers, and the interop-script template points at the concrete test files rather than re-inventing a smaller example. - cli/README.md — the "For an interop-test script template" pointer now links to `cli/tests/README.md` alongside the DEVELOPMENT.md section. - .claude/skills/amy-expert/SKILL.md — the "Where things live" tree omitted the new `tests/` subtree entirely. Added it with a one-line description per suite. Also extended the wire-up checklist with a step 7: add a harness case when a new verb changes observable wire behaviour. No content changes elsewhere; just path corrections. --- .claude/skills/amy-expert/SKILL.md | 13 ++++++++++++- cli/DEVELOPMENT.md | 9 +++++++-- cli/README.md | 5 ++++- 3 files changed, 23 insertions(+), 4 deletions(-) diff --git a/.claude/skills/amy-expert/SKILL.md b/.claude/skills/amy-expert/SKILL.md index bf423a025..1677296d1 100644 --- a/.claude/skills/amy-expert/SKILL.md +++ b/.claude/skills/amy-expert/SKILL.md @@ -104,9 +104,15 @@ Wire-up checklist: 4. Extend `printUsage()` in `Main.kt`. 5. Add the row to `cli/README.md`'s command table. 6. Update `cli/ROADMAP.md` — move the row from 🆕 / 📦 to ✅. +7. If the verb changes observable wire behaviour (a new event kind, + a new relay-routing rule, a new JSON discriminator), add a case + in the appropriate harness under `cli/tests/` — `cli/tests/marmot/` + for MLS flows, `cli/tests/dm/` for NIP-17, or a new sibling suite + if it's neither. If you change output shape: note it in the commit message, bump the -example in `README.md`, update any interop fixtures. +example in `README.md`, update any interop fixtures under +`cli/tests/`. ## Where things live @@ -116,6 +122,11 @@ cli/ ├── DEVELOPMENT.md # touch-the-code: architecture, conventions, testing ├── ROADMAP.md # parity matrix + ordered milestones ├── plans/ # dated design docs (use for new subsystems) +├── tests/ # end-to-end shell harnesses against a local relay +│ ├── lib.sh # shared logging + result tracking +│ ├── headless/ # shared amy wrappers + assertions +│ ├── marmot/ # MLS group-messaging interop (vs whitenoise-rs) +│ └── dm/ # NIP-17 DM interop (two amy clients) └── src/main/kotlin/…/cli/ ├── Main.kt # argv dispatch ├── Args.kt # flag parser diff --git a/cli/DEVELOPMENT.md b/cli/DEVELOPMENT.md index 2d5b990c2..6b855761a 100644 --- a/cli/DEVELOPMENT.md +++ b/cli/DEVELOPMENT.md @@ -197,8 +197,8 @@ Amy-specific layer still needs its own coverage: | Error / exit-code contract (bad args → 2, await timeout → 124, runtime → 1) | Table-driven tests invoking `main(argv)` with captured stdout/stderr. | | JSON output shape (each command's keys and types) | Snapshot tests: run a command against a throwaway data-dir, assert the JSON matches a golden file. | | File layout on disk (`identity.json`, `relays.json`, `groups/*.mls`, `keypackages.bundle`) | Structural assertions after a command sequence. | -| Round-trip between two data-dirs on a local relay | End-to-end shell scripts under `cli/src/test/resources/scripts/`. Spin up `nostr-rs-relay`, run Alice + Bob, assert await verbs resolve. | -| Interop with other clients | External harness consumes Amy as a binary; out of scope here but the JSON contract is what keeps it stable. | +| Round-trip between two data-dirs on a local relay | End-to-end shell harnesses under `cli/tests/`. Each harness spins up a local `nostr-rs-relay`, bootstraps two or more fresh identities in their own `--data-dir`s, and drives a scenario via `amy` (+ `wn` for Marmot interop against whitenoise-rs). Today there are two suites: `cli/tests/marmot/` (13 MLS scenarios vs whitenoise-rs) and `cli/tests/dm/` (NIP-17 DM round-trips between two `amy` clients). | +| Interop with other clients | Covered by `cli/tests/marmot/marmot-interop-headless.sh` (drives Amy against whitenoise-rs `wn`/`wnd`). Add new scenarios there or start a new sibling under `cli/tests/`. | **What not to test here:** event signing, filter assembly, MLS correctness, NIP-44 encryption. Those belong in `quartz`/`commons`. @@ -207,6 +207,11 @@ If an Amy bug can only be caught here, it's a contract violation **Interop-test script template:** +The canonical examples live under `cli/tests/` — read +[`cli/tests/README.md`](./tests/README.md) for the layout, then +crib from `cli/tests/dm/tests-dm.sh` or `cli/tests/marmot/tests-create.sh`. +At the byte-banging level, a minimal round-trip looks like: + ```bash set -euo pipefail TMP=$(mktemp -d) diff --git a/cli/README.md b/cli/README.md index a1077f37e..abc214147 100644 --- a/cli/README.md +++ b/cli/README.md @@ -98,7 +98,10 @@ GID=$(amy --data-dir ./alice marmot group create --name "Test" | jq -r .group_id ``` For an interop-test script template, see -[DEVELOPMENT.md § Testing](./DEVELOPMENT.md#testing). +[DEVELOPMENT.md § Testing](./DEVELOPMENT.md#testing). The runnable +harnesses live under [`cli/tests/`](./tests/README.md) — +`cli/tests/marmot/` for MLS group messaging vs whitenoise-rs, +`cli/tests/dm/` for NIP-17 DMs between two `amy` clients. ---