test(nests): add manual interop harness against nostrnests.com

Adds cli/tests/nests/nests-interop.sh — a 47-test operator-driven
interop script that walks a tester through every audio-room edge case
between Amethyst Android and the nostrnests.com NestsUI-v2 web client.

Coverage: bidirectional host/listener flows, audio round-trip on
moq-lite Lite-03, hand-raise + role promotion/demotion, mute, kind:7
reactions (including NIP-30 custom emoji + 30 s overlay clear), kind:1311
in-room chat (text + emoji + link + image upload + history backfill),
kind:4312 kick, room edit + close, scheduled rooms (status=planned),
multi-speaker, background audio + PIP, network drop reconnect, 10-min
JWT refresh, custom moq servers (kind:10112), naddr deep links, and
edge cases (long titles, empty rooms, leave-stage, dedupe, race).

The script follows the marmot/marmot-interop.sh pattern: shared
logging + result helpers, color-coded prompts (yellow=Amethyst,
magenta=web, cyan=optional 3rd identity), p/f/s confirms recorded to a
TSV, and a final summary table. Skip / only / keep-state flags are
supported for iteration.

Pure manual harness — amy does not yet ship `amy nests <verb>` so
nothing is automatable from the CLI side.
This commit is contained in:
Claude
2026-04-28 07:52:57 +00:00
parent 2027f6ad37
commit 88c158433c
4 changed files with 1655 additions and 4 deletions
+1
View File
@@ -1,3 +1,4 @@
marmot/state/
marmot/state-headless/
dm/state-dm-headless/
nests/state/
+18 -4
View File
@@ -16,10 +16,13 @@ cli/tests/
│ ├── tests-manage.sh # tests 0608, 11
│ ├── tests-extras.sh # tests 09, 10, 12, 13
│ └── patches/ # whitenoise-rs harness patches
── dm/ # NIP-17 DM interop (amy ↔ amy)
├── dm-interop-headless.sh
├── setup.sh # preflight + identities
└── tests-dm.sh
── dm/ # NIP-17 DM interop (amy ↔ amy)
├── dm-interop-headless.sh
├── setup.sh # preflight + identities
└── tests-dm.sh
└── nests/ # Audio-rooms interop (Amethyst ↔ nostrnests.com)
├── nests-interop.sh # 47-test manual harness
└── README.md # operator brief + per-test matrix
```
The Marmot harnesses come in two flavours, same scenarios:
@@ -41,6 +44,17 @@ A third, slimmer harness covers the NIP-17 DM surface:
is shared with the Marmot harness's checkout at
`marmot/state-headless/nostr-rs-relay/`).
A fourth harness covers audio rooms (NIP-53 + moq-lite):
- **`nests/nests-interop.sh`** — fully manual interop between Amethyst
Android and the [nostrnests.com](https://nostrnests.com) reference
web client. 47 tests spanning host/listener flows, audio round-trip,
hand-raise + role promotion, reactions, in-room chat (kind 1311),
kick (kind 4312), close-room, schedule, network-drop reconnect,
10-min JWT refresh, custom moq servers (kind 10112), and PIP /
background audio. See `nests/README.md` for the full matrix and
prereqs.
Both Marmot harnesses validate Amethyst against **whitenoise-rs**
(https://github.com/marmot-protocol/whitenoise-rs), the reference Rust
implementation that powers the White Noise Flutter app. Every test records a
+200
View File
@@ -0,0 +1,200 @@
# Nests audio-room interop harness
Manual interop test script for Amethyst's audio-rooms feature against
[nostrnests.com](https://nostrnests.com) (the reference web client
[`NestsUI-v2`](https://github.com/nostrnests)). Verifies the full
NIP-53 / kind:30312 + moq-lite Lite-03 stack end-to-end, prompting the
operator at every step.
This harness is **fully manual**`amy` does not currently ship
`amy nests <verb>` subcommands, so every action runs through the
production UIs on each side. The script's job is to walk the operator
through every feature + edge case and record pass / fail / skip per
test, the same way `marmot/marmot-interop.sh` does for MLS.
## Quick start
```bash
cd cli/tests/nests
./nests-interop.sh
```
The script prompts for two npubs (Amethyst account A, web account W)
and an optional third (X, used by tests 11, 28, 32, 45, 47), then
walks through 47 tests covering every interop surface.
To run a subset:
```bash
./nests-interop.sh --only 03 --only 10 # just audio-path round-trips
./nests-interop.sh --skip 36 --skip 38 # skip the 11- and 6-minute waits
./nests-interop.sh --keep-state # reuse cached npubs from a prior run
```
## Prerequisites
### Amethyst (account A)
1. Build & install **this branch** (`claude/test-nests-amethyst-interop-WETiY`)
on a real Android device (emulator works for most tests but not
tests 14 / 16 / 18 / 33 / 34 — mic capture, foreground service, PIP).
2. Sign in with account A.
3. **Settings → Relays** — make sure the relay set overlaps the web
client's. Default `damus.io / nos.lol / primal.net` works for
nostrnests.com out of the box.
4. **Settings → Audio-room servers** — confirm
`nostrnests.com → https://moq.nostrnests.com` is present (default).
5. Grant **microphone** + **post notifications** permissions on first
prompt (the foreground-service notification needs the latter on
Android 13+).
### nostrnests.com (account W)
1. Open <https://nostrnests.com/> in a Chromium-based browser
(Chrome, Edge, Brave) — Firefox does not yet ship WebTransport
stable.
2. Install a Nostr signer extension — [Alby](https://getalby.com),
[nos2x](https://github.com/fiatjaf/nos2x), or similar.
3. Sign in with account W via the extension popup.
4. Allow microphone access when the browser prompts.
5. Confirm the relay list overlaps Amethyst's.
### Optional third identity (X)
Used by:
| Test | Why X is needed |
|---|---|
| 04, 11 | Verifies the listener counter dedupes correctly with 2 listeners |
| 28 | Late-join chat-history backfill |
| 32 | 3-speaker concurrent broadcast |
| 45 | Same-pubkey-on-two-devices dedupe (X is the second device) |
| 47 | Concurrent host-edit race |
Provide X via either nostrnests.com in a second browser profile, or a
second Amethyst install on a different device. If you skip X the
dependent tests are recorded as `SKIP` and the rest run normally.
## What is covered
| # | Test | Surface |
|---|---|---|
| 01 | Amethyst hosts → web discovers | kind:30312 publish + relay scan |
| 02 | Web joins as listener | NIP-98 → JWT mint → moq-lite Lite-03 ALPN |
| 03 | Amethyst → web audio | Opus encode → moq uni-stream → Web Audio decode |
| 04 | Listener counter aggregation | kind:10312 dedupe by pubkey |
| 05 | Edit room metadata | kind:30312 re-publish round-trip |
| 06 | Web leaves | presence with `["leaving"]`, then heartbeat decay |
| 07 | Amethyst host closes | kind:30312 with `["status","ended"]` |
| 0811 | Reverse direction (web hosts) | symmetric with 0104 |
| 12 | Listener cannot publish | role-gating in moq-auth + client UI |
| 1314 | Mute round-trips | kind:10312 with `["muted","1"]` |
| 1519 | Hand-raise + promote / demote | kind:10312 `["hand","1"]` + kind:30312 role re-publish |
| 2022 | Reactions (kind 7) | NIP-25 + NIP-30 custom emoji |
| 23 | Reaction overlay auto-clear | 30-second visibility window |
| 2428 | In-room chat (kind 1311) | including image upload + history backfill |
| 2930 | Kick (kind 4312) | ephemeral admin command + auto-disconnect |
| 31 | Web closes room | reverse of test 07 |
| 32 | Multi-speaker | 3 concurrent moq-lite publishers |
| 33 | Background audio | `AudioRoomForegroundService` |
| 34 | PIP screen | `NestPipScreen` |
| 35 | Network drop → reconnect | `ReconnectingNestsListener` + backoff |
| 36 | Long session (11 min) | JWT re-mint past 10-min `exp` |
| 37 | Force-close | ghost-listener decay via heartbeat-timeout |
| 38 | Scheduled rooms | `["status","planned"]` + `["starts","<unix>"]` |
| 39 | Themed room | EGG-10 graceful fallback |
| 40 | Share via naddr | NAddress deep link round-trip |
| 41 | Custom moq server | kind:10112 `NestsServersEvent` |
| 42 | Empty room (host alone) | publisher path with 0 listeners |
| 43 | Long title / summary | UI truncation + relay size limits |
| 44 | Leave stage (onstage=0) | role tag preserved across stage exit |
| 45 | Same-pubkey two-device dedupe | counter sees one logical user |
| 46 | Per-participant actions | profile / follow / mute / zap from a room |
| 47 | Concurrent host-edit race | last-write-wins on relay |
Tests 36 and 38 contain `sleep` calls (11 min and 6 min). Skip them
with `--skip 36 --skip 38` if you're iterating.
## How human interaction works
Three prompt styles. The color always tells you which side to act on:
```
---- DO THIS IN AMETHYST (Android) ---- # yellow
1. Tap …
[Press Enter when done]
```
```
---- DO THIS ON nostrnests.com (web) ---- # magenta
1. Click …
[Press Enter when done]
```
```
---- DO THIS ON THE 3RD IDENTITY (X) ---- # cyan
1. …
```
After each test the script asks a verification question:
```
? Did Amethyst's Audience now show W?
[p]ass / [f]ail / [s]kip:
```
Pick `p`, `f`, or `s`. The result is logged + tallied in the final
summary. Fails record `FAIL <test-id>` to the log so you can grep
later. Skipped tests carry a note explaining why (filtered, no third
identity, operator declined long-wait, etc.).
## Output
- `state/logs/run-<timestamp>.log` — every prompt, every confirm
outcome, every value the operator pasted.
- `state/results-<timestamp>.tsv``test_id <TAB> pass|fail|skip <TAB> note`.
- `state/run.env` — cached npubs (and any other state captured via
`prompt_text`); re-used with `--keep-state`.
- Final colored summary table on stdout.
## Cleanup
State is kept on disk so you can `--keep-state` to resume across runs.
To start completely fresh:
```bash
rm -rf cli/tests/nests/state/
```
Rooms created during a run are real public rooms on
nostrnests.com. They're closed automatically by the script (see
test 07, 31, etc.) but if you `Ctrl-C` mid-run, leftover rooms will
auto-close after ~8 hours of inactivity per the nests server's
default policy.
## Known limitations
- **Audio quality** is operator-judged ("did you hear it cleanly?").
No automatic SNR / latency measurement.
- **Tests 33 / 34** require a real Android device — emulators don't
expose the foreground-service notification or PIP gestures the
same way as a phone.
- **Test 36** (11-min token refresh) and **test 38** (6-min schedule
wait) are slow. Use `--skip 36 --skip 38` during iteration.
- **Test 45** needs a second Android device or emulator running A.
- **Test 41** needs a non-default moq-rs deployment to be meaningful.
- The script does **not** verify which moq endpoint a connection
actually used — that visibility lives in
`nestsClient/src/jvmTest/interop/` (the Docker-driven JVM tests
gated by `-DnestsInterop=true`). Pair this script with that suite
for a complete picture.
## Pointers
- `nestsClient/specs/README.md` — EGG conformance specs
- `nestsClient/plans/2026-04-26-audio-rooms-completion.md` — feature status
- `nestsClient/plans/2026-04-26-nostrnests-integration-audit.md` — gap list
- `nestsClient/plans/2026-04-26-moq-lite-gap.md` — wire-spec details
- `cli/tests/marmot/marmot-interop.sh` — the inspiration / pattern this script follows
- Amethyst nests UI: `amethyst/src/main/java/com/vitorpamplona/amethyst/ui/screen/loggedIn/nests/`
- nests transport / audio: `nestsClient/src/commonMain/`
+1436
View File
File diff suppressed because it is too large Load Diff