Adds cli/tests/nests/nests-interop.sh — a 47-test operator-driven
interop script that walks a tester through every audio-room edge case
between Amethyst Android and the nostrnests.com NestsUI-v2 web client.
Coverage: bidirectional host/listener flows, audio round-trip on
moq-lite Lite-03, hand-raise + role promotion/demotion, mute, kind:7
reactions (including NIP-30 custom emoji + 30 s overlay clear), kind:1311
in-room chat (text + emoji + link + image upload + history backfill),
kind:4312 kick, room edit + close, scheduled rooms (status=planned),
multi-speaker, background audio + PIP, network drop reconnect, 10-min
JWT refresh, custom moq servers (kind:10112), naddr deep links, and
edge cases (long titles, empty rooms, leave-stage, dedupe, race).
The script follows the marmot/marmot-interop.sh pattern: shared
logging + result helpers, color-coded prompts (yellow=Amethyst,
magenta=web, cyan=optional 3rd identity), p/f/s confirms recorded to a
TSV, and a final summary table. Skip / only / keep-state flags are
supported for iteration.
Pure manual harness — amy does not yet ship `amy nests <verb>` so
nothing is automatable from the CLI side.
androidx.sqlite SQLiteConnection is not thread-safe; SQLiteEventStore
shared a single lazy connection across all callers, so two coroutines
calling insertEvent() at the same time would race on BEGIN IMMEDIATE
and the modules' prepared statements, surfacing as
"cannot start a transaction within a transaction" or SQLITE_MISUSE.
Mirror Room's design: introduce SQLiteConnectionPool with one writer
connection guarded by a coroutine Mutex and N reader connections
handed out via a Channel-as-semaphore (file-backed DBs only; in-memory
DBs share the writer because each ":memory:" connection is a separate
DB). Convert IEventStore + SQLiteEventStore + EventStore + FsEventStore
+ LiveEventStore to suspend, route writes through useWriter and reads
through useReader. RelaySession now launches handleEvent / handleCount
on its scope. CLI Context helpers and StoreCommands.sweepExpired pick
up suspend.
Add ParallelInsertTest to lock the behaviour in: 8 coroutines × 200
inserts, parallel reads alongside writes, transaction batches across
coroutines, and a reopen smoke test all pass against a file-backed DB.
https://claude.ai/code/session_016b5kSSbtDS3Ead6pN3Xqt5
USAGE.md was the better README — entry-point users want examples and
quick start, not the public-API contract. Flip them and refresh the
amy-expert skill so it matches the post-refactor reality.
cli/README.md (was USAGE.md):
- Install, quick start, seven worked examples, full command reference,
output modes, multi-account workflows, agent recipes, troubleshooting.
- Cross-refs point at DEVELOPMENT.md for the contract / architecture
and ROADMAP.md for what's coming.
cli/DEVELOPMENT.md absorbs the old README's architecture sections:
- New "Public contract" section at the top — the stable promises
(text-default + --json contract, stderr for humans, exit codes,
~/.amy/ as the world).
- "Local event store" deep-dive with the cache-helper API.
- "Relay routing" rules table.
- "Full on-disk layout" tree with annotations.
cli/ROADMAP.md, cli/USAGE.md:
- ROADMAP cross-refs collapsed (no more USAGE.md row).
- USAGE.md deleted — content lives in README now.
.claude/skills/amy-expert refreshed end-to-end:
- SKILL.md description + Rules 2 and 4 rewritten for the dual-output
contract (text default, --json opt-in) and the ~/.amy/ layout.
- "Where things live" listing matches the current source tree
(Output.kt, Aliases.kt, UseCommand.kt, secrets/, all the new
command files).
- "Common mistakes" lists the new traps: don't read user.home
directly, don't add a global flag that collides with subcommand
--name, don't use Json.writeLine (it's gone).
- references/command-template.md uses Output.emit / Output.error
(Json.writeLine / Json.error helpers no longer exist).
- references/output-conventions.md rewritten around the dual-mode
contract — same JSON shape rules, but framed as "this is what
--json emits" rather than "this is stdout."
Quartz's RelayUrlNormalizer.isLocalHost strips literal 127.0.0.1 /
localhost / 192.168.* / .local / umbrel out of NIP-17 inbox
(kind:10050) and KeyPackage (kind:10051) relay-list events as a
privacy guard. The marmot harness was binding the loopback relay to
ws://127.0.0.1:8080, so when amy added that URL to its kind:10050 /
kind:10051 events the parser silently dropped it on read — leaving
the harness publishing to Amethyst's PUBLIC default relays
(nos.lol, nostr.mom, …) instead of the local one. Whitenoise (which
only listens on the local socket) never saw amy's KeyPackage,
breaking every Test 01–16 scenario at the first hop.
The DM harness already worked around this by using 127.0.0.2 (still
pure loopback, not on the strip list — see dm-interop-headless.sh:34).
This commit applies the same workaround to the marmot harness:
- Default RELAY_HOST=127.0.0.2 (overridable via env or --host).
- RELAY_URL is now derived from RELAY_HOST + RELAY_PORT.
- New --host flag for parity with the DM harness.
setup.sh's relay config already reads RELAY_HOST when binding
nostr-rs-relay (line 173, address = "${RELAY_HOST:-127.0.0.1}"), so
no other change is needed — the relay binds where amy is trying to
reach it.
USAGE.md is now the single home for "how do I use amy" — install,
quick start, examples, command reference, troubleshooting. README
was carrying all of that PLUS the public-contract material; with
USAGE in place it can collapse to just the contract.
README.md (338 → 176 lines):
- Keep: 1-paragraph intro (three audiences), output contract, local
event-store explainer, relay-routing rules, on-disk layout, cross-
references to USAGE/DEVELOPMENT/ROADMAP.
- Drop: install, quick start, command reference table, global flags
table, account-management verbs, troubleshooting (all in USAGE).
DEVELOPMENT.md:
- Architecture diagram updated to reflect new files (Aliases.kt,
UseCommand.kt, ProfileCommands.kt, DmCommands.kt, NotesCommands /
PostCommand / FeedCommand, MarmotResetCommand, StoreCommands,
SecureFileIO, secrets/ subtree).
- "Keep three things in sync" pointers redirected from README's
command table to USAGE.md.
- Testing table loses the duplicate "Interop with other clients" row
(already covered by the harness row above).
- Cross-reference list at the top now mentions USAGE.md.
ROADMAP.md:
- Cross-reference list adds USAGE.md.
- Parity matrix marked ✅ for items shipped on this branch:
notes post + feed (PostCommand/FeedCommand), profile show+edit
(ProfileCommands), DMs (already ✅).
- Reactions split into "✅ in groups, 🆕 elsewhere" since
marmot message react is shipped but outer-event reactions aren't.
- Order-of-operations entries marked ✅ where relevant.
Modeled on nostrcli.sh's docs page: quick start, seven worked examples
(post a note, send a DM, read a thread, view a profile, MLS group
round-trip, account switching, add a relay), categorised command
reference, output-modes section, multi-account workflow, agent/script
recipes, troubleshooting.
README.md and DEVELOPMENT.md are referenced from here; they stay
focused on the public contract and the extension rules.
The marmot harness surfaced the bug on first run: amy stripped
`marmot group create --name "Interop-02"` thinking the global
account selector ran into the group's display-name flag. Result:
amy resolved the "Interop-02" account, found no identity.json, and
errored — no group ever got created.
Renames the global account-selector flag to `--account`. The per-
subcommand `--name` flags (`marmot group create --name "Demo"`,
`profile edit --name "Alice"`, `marmot await group --name X`) are
untouched — they're free of the global parser now that it doesn't
claim the same name.
Sweep:
- `Main.kt`: GlobalFlag.NAME → ACCOUNT, long "--account".
- `Config.kt`: DataDir.resolve param renamed nameFlag → accountFlag;
every error message points the user at --account.
- `UseCommand.kt`: error hint says `amy --account NAME init`.
- All test wrappers + direct $AMY_BIN calls under cli/tests/ swap
the global `--name X` for `--account X` (subcommand --name kept
exactly where it appeared).
- README + DEVELOPMENT updated.
There is no installed base to protect, so the self-contained
`--data-dir P` escape hatch goes away entirely. amy now has exactly
one layout — the production multi-account one — and tests exercise
that same code path. Drops one global flag, one DataDir construction
mode, and the "tests use a different code path than users" footgun.
Layout (unchanged from the previous commits — just the only mode now):
~/.amy/
├── current # `amy use NAME` marker
├── shared/
│ └── events-store/ # FsEventStore, one per machine
├── alice/
│ ├── identity.json
│ ├── state.json
│ ├── aliases.json # {"alice": "<own npub>"} after init
│ └── marmot/
└── bob/ ...
`DataDir.DEFAULT_ROOT` reads `$HOME` directly (falling back to
`user.home`) because JDK 21 resolves `user.home` via getpwuid and
ignores `$HOME` — which would have broken the standard CLI test
isolation pattern of `HOME=/tmp/foo amy …` (the same convention
git, gpg, npm follow).
Test sweep:
- `cli/tests/headless/helpers.sh`, `cli/tests/dm/setup.sh`,
`cli/tests/cache/cache-headless.sh` wrappers all switch to
`HOME=$STATE_DIR amy --name X …`.
- `ensure_identity_for` drops its `dir` parameter; the function and
every harness call site go through `--name` only.
- `A_DIR`/`B_DIR`/`D_DIR` get repointed at `$STATE_DIR/.amy/X`. The
one consumer (`cache-headless.sh`'s T6 `relays.json` check) still
works since it's just a path probe.
- `cli/tests/dm/tests-dm.sh` ghost identities get their own
short-lived `$HOME` so they don't pollute the main test root.
Cache-test T4 inverts: pre-shared-store, "B has not seen A → first
profile show is a relay miss, second is a cache hit" tested
per-account caching. With one shared events-store, B's first lookup
of A is already a cache hit because A wrote kind:0 there during
bootstrap. T4 now asserts that — drops the stale "second lookup hits
cache" half. T7 (no-identity maintenance verbs) gets a fresh fake
`$HOME` plus a throwaway `--name` so the empty store has no inherited
events.
Docs (README + DEVELOPMENT) rewritten to match — quick-start now uses
`amy --name alice create`, the on-disk-layout section shows the new
tree, and the global-flags table replaces `--data-dir` with `--name`
plus the new `amy use` verb.
Resolution order in account mode (when --data-dir is not set):
1. --name X if given.
2. ~/.amy/current marker (set by `amy use X`).
3. Sole subdirectory of ~/.amy/ other than shared/.
4. Error — disambiguate with --name or `amy use`.
Single-account users skip the flag entirely (`amy whoami` Just Works
once one account exists). Multi-account users either pin one with
`amy use bob` (writes ~/.amy/current) or pass --name on every call.
The pinned account can be overridden by --name on a single command,
and cleared with `amy use --clear`.
`amy use` (no arg) prints the current pin plus the list of available
accounts. `amy use NAME` validates the name, requires the account dir
to already exist (else `no_account` with a creation hint), and
atomically writes the marker.
The auto-pick errors are deliberately self-explaining:
- 0 accounts → "no account at ~/.amy; create one with `amy --name X init`"
- 2+ accounts unpinned → "multiple accounts (alice, bob); pick one
with --name or `amy use <name>`"
- stale current marker → "current pins 'ghost' but ~/.amy/ghost
doesn't exist; rewrite with `amy use <name>` or pass --name"
`use` is dispatched before DataDir.resolve so it works even when the
auto-pick would fail — that's the whole point of having the verb.
The `name` field also lands in `whoami` output now (null in
--data-dir legacy mode).
Default on-disk layout becomes:
~/.amy/
├── shared/
│ └── events-store/ (lazy: created on first event)
└── <account>/ (created by `amy --name X init`)
├── identity.json
├── state.json
├── aliases.json
└── marmot/
Per-account state moves under `~/.amy/<account>/`; the events-store is
shared across accounts under `~/.amy/shared/`. The shared store is
safe to share for now because amy doesn't currently persist any
decrypted inner events to it (NIP-17 DMs unwrap-and-display in
DmCommands; MLS inner events go to the per-group .log under
<account>/marmot/groups/, not the event store). When that changes,
a follow-up will introduce a per-account private-events-store and a
composite reader.
A new `--name X` global flag selects (or creates) `~/.amy/X/`.
`--data-dir P` is preserved as a self-contained escape hatch — the
test harness and ad-hoc throwaway dirs use it, events-store stays
inside P. Pass exactly one of `--name` or `--data-dir`; passing both
or neither is bad_args (exit 2). Names must match
[a-zA-Z0-9_-]{1,64}; `shared` is reserved.
`amy init --name alice` writes a self-entry into
`<account>/aliases.json` ({"alice":"npub1…"}) so future commands and
the planned `amy alias add` / dm-recipient resolver can refer to the
account by name. The init result map gains a `name` key (null in
legacy mode).
Open question for a follow-up: when only one account exists in
~/.amy/, should `amy whoami` work without --name? Today it doesn't —
strict mode. Also pending: README rewrite, plus a sweep through
commands that print `data_dir` to also surface `name` where useful.
`profile show` puts the parsed kind:0 content under `metadata` as a
Jackson `JsonNode` (`ProfileCommands.kt:114`). The text renderer only
recursed into `Map`/`List`, so the JsonNode fell through to
`toString()` and printed as a single quoted-JSON line:
metadata: {"name":"Alice","picture":"…",…}
Convert any embedded JsonNode to plain Java types via
`mapper.convertValue` once at the top of `renderText`, so the same
generic walk yields:
metadata:
name: Alice
picture: …
…
The walk handles nested cases (a hand-built Map containing a JsonNode
subtree, an ArrayNode inside a List, etc.). The `--json` shape is
untouched — Jackson's `writeValueAsString` already serialises JsonNode
natively.
Defaults to ANSI colour when stdout is a TTY (off when piped, off under
NO_COLOR=…, force-on under CLICOLOR_FORCE=1). Layout improvements that
apply to every command without per-command code:
- Sibling keys at each indentation level pad to the widest, so colons
line up YAML-style.
- Keys render bold; list dashes and "(none)"/"(empty)" markers render
dim.
- Booleans render as `yes` / `no` (green / red).
- Integer values under `*_at` keys render as
`2026-04-25 12:30:45Z (2m ago)` instead of a raw epoch second.
- Integer values under `*_bytes` keys (and `size`) render as
`7.0 KiB` / `1.2 MiB` / etc.
- Errors render `error: <code>: <detail>` with the prefix bold-red and
the code yellow.
The `--json` shape and exit codes are untouched; the smart formatting
only changes how scalars surface in the human-text mode.
amy's default stdout is now a YAML-ish render of the underlying result
map; the previous single-line JSON contract moves behind a global
`--json` flag. Errors mirror the same rule (`error: <code>: <detail>`
on stderr by default, JSON `{"error":...,"detail":...}` under --json).
Exit codes (0/1/2/124) and the --json shape itself are unchanged —
only the default presentation flips.
- Replaces Json.writeLine / Json.error with mode-aware
Output.emit / Output.error. The same Jackson mapper is reused for
on-disk JSON via Output.mapper.
- Adds `--json` to the global flag set in Main.kt; honoured even when
argument parsing fails so error JSON keeps shape under --json.
- Updates the test harness wrappers (amy_a / amy_b / amy_d in
cli/tests/{headless,dm,cache}/) and the few direct $AMY_BIN call
sites whose stdout is consumed via $() / 2>&1 — they now pass
--json so the existing jq pipelines keep working.
- Rewrites the README "Output contract" and DEVELOPMENT design
principles to describe the new default, and clarifies that only
--json is the public API; the text shape is allowed to drift.
assert_eq is (actual, expected, test_id, note); first version of the
script had test_id and actual swapped, so even passing assertions
showed up as fails with confusing "got T2.source" messages. Now all
21 assertions pass with the right arg order. Each successful assertion
also fires record_result pass so the results table covers them
(previously the tests passed but were invisible — only the explicit
record_result calls showed up).
Verified end-to-end on a real local nostr-rs-relay:
21 passed, 0 failed, 0 skipped (of 21)
Coverage:
T1.* — store stat reports kind histogram + disk usage after publish
T2.* — self profile show is served from cache by default
T3.* — --refresh forces source: relays
T4a/b — B's first lookup of A is a relay miss; second is a cache hit
T5.* — relay list reads URLs back from local kind:10002/10050/10051
T6 — relays.json is gone from both data-dirs
T7.* — store maintenance verbs work without an identity
Three changes that go together:
1. Reconcile cli/plans/2026-04-24-file-event-store-{overview,nips}.md
with shipped reality: code lives in quartz/jvmMain/, not commons/;
data dir is <data-dir>/events-store/, not <root>/events/.
2. New StoreCommands wired as `amy store …`:
- stat → events count, kind histogram, disk bytes,
oldest/newest createdAt. Pure read, no Context
(skips identity check).
- sweep-expired → wraps store.deleteExpiredEvents(); reports
{swept, remaining}.
- scrub → wraps store.scrub() to rebuild idx/ from
canonicals.
- compact → wraps store.compact() to drop dangling idx/.
All four open the FsEventStore directly (no Context, no identity
needed) — they're store-only operations.
3. New e2e harness at cli/tests/cache/cache-headless.sh that boots a
local nostr-rs-relay, two amy identities (A + B), and asserts:
T1 — store stat reports non-empty store after publish-lists +
profile edit, with kind:0 and kind:10002 present.
T2 — A's `profile show` is `source: "cache"` by default.
T3 — `--refresh` forces `source: "relays"`.
T4 — B's first `profile show <A_NPUB>` is a relay miss; second is
a cache hit (proves drain populates the local store and
subsequent reads serve from disk).
T5 — `relay list` reads URLs back from the local kind:10002 /
10050 / 10051 events.
T6 — relays.json no longer exists in either data-dir.
T7 — store stat / sweep-expired / scrub / compact all run
without an identity present.
Same pattern as cli/tests/dm/dm-interop-headless.sh — reuses the
nostr-rs-relay infrastructure from cli/tests/marmot/setup.sh.
Users actually look at <data-dir>/events-store/ files (cat / jq / git
diff), so the CLI now writes them with the InliningTagArrayPrettyPrinter
that quartz already had configured but never invoked. Each event is
indented with 2 spaces, but every tag array stays on a single line —
nice trade-off between human-readable and not-too-tall:
{
"id": "...",
"pubkey": "...",
"created_at": 1700000000,
"kind": 1,
"tags": [
["t","nostr"],
["e","abc...a"],
["alt","quick brown fox"]
],
"content": "hi there",
"sig": "..."
}
Stored bytes are not the canonical NIP-01 form — but verification
re-canonicalises via EventHasher anyway, so format is purely a UX
choice. Compact stays the default for any caller that doesn't opt
in (Android keeps SQLite, generic FsEventStore embedders keep
compact).
- JacksonMapper.toJsonPretty(event): new entry point that uses
writerWithDefaultPrettyPrinter() with the existing inlining printer.
- FsEventStore now takes an `eventToJson: (Event) -> String` callback,
default = Event::toJson (compact). Used in insert.
- Context wires in JacksonMapper::toJsonPretty.
2 new tests in FsEventToJsonTest pin both formats (compact stays
single-line; pretty round-trips). 117 fs tests green.
The local relay configuration was redundant once the event store
became the source of truth: every account already publishes signed
kind:10002 (NIP-65), kind:10050 (DM inbox), and kind:10051
(KeyPackage relays) events, and Context now reads each set straight
out of the local store.
Removed:
- data class RelayConfig (nip65/inbox/keyPackage buckets)
- DataDir.relaysFile / loadRelays() / saveRelays()
- Context.relays parameter — Context.open() no longer reads JSON
- CreateCommand.defaultRelayConfig() helper
- CreateCommand's saveRelays() call (redundant: ctx.publish persists
the bootstrap events into the store anyway)
Context now serves the four relay accessors from the store with sane
fallbacks:
outboxRelays() = relaysOf(self)?.writeRelaysNorm() ?: DefaultNIP65RelaySet
inboxRelays() = dmInboxOf(self)?.relays() ?: DefaultDMRelayList.toSet()
keyPackageRelays() = keyPackageRelaysOf(self)?.relays() ?: outboxRelays()
anyRelays() = union of the three
bootstrapRelays() = anyRelays() ∪ DefaultNIP65RelaySet ∪ DefaultDMRelayList
RelayCommands rewritten:
- `relay add URL --type T` — read existing relay-list event from the
store, append URL, build + sign + ingest a new event of the
matching kind. The replaceable slot mechanism replaces the old
winner atomically.
- `relay list` — dump URLs from the latest event in each bucket.
- `relay publish-lists` — broadcast whichever events are present in
the store; errors out cleanly when none exist (suggests `relay add`
or `create`).
No migration: existing data dirs that still have `relays.json` will
ignore it. Run `amy relay add` or `amy create` to repopulate. Per
discussion this is a clean break, not a backwards-compat shim.
README + DEVELOPMENT updated to reflect the new on-disk layout (no
more `relays.json`; events-store/ is the relay configuration).
Audit found four commands draining replaceable relay-list events
(kind:10050, 10051, 10002) on every invocation, plus one fetching
the user's own kind:3 contact list. All five are now cache-first.
New Context helpers:
- dmInboxOf(pk) → kind:10050 (ChatMessageRelayListEvent)
- keyPackageRelaysOf(pk)→ kind:10051 (KeyPackageRelayListEvent)
- cachedRelayListsOf(pk)→ assembles a RecipientRelayFetcher.Lists
from the local store; returns null if
nothing is cached so callers can fall
back with `?:`.
Wired into:
- FeedCommand.resolveFollowing — replaces a kind:3 drain with
ctx.contactsOf(self).
- DmCommands.resolveDmRelays — `amy dm send` / `dm list`.
- KeyPackageCommands.check.
- AwaitCommands (key-package / member / admin polling loops).
- GroupAddMemberCommand (one cache hit per invitee).
All five sites now go: try cache → if miss, run the existing
RecipientRelayFetcher / drain. The drain itself populates the cache
via verifyAndStore, so subsequent runs are local-only. Replaceable
slot shortcut means each cached read is one stat + one readString,
not a directory walk.
README "Local event store" section gains a list of which commands
are cache-first today.
The first command to consume the local event store as a read source.
Default behaviour: read from the FsEventStore via Context.profileOf().
Pass --refresh to skip the cache and drain from relays (which then
re-populates the store as a side effect, like every other drain).
Output JSON gains a "source" field so scripts/agents can tell whether
the result came from disk or the network:
source = "cache" → served from <data-dir>/events-store/, queried_relays = []
source = "relays" → fresh relay drain, queried_relays = list
Cache hit goes through the planner's slot shortcut from the previous
commit: one stat + one read on replaceable/0/<pubkey>.json, no
directory scan.
profile show + profile edit now have entries in the verb table in
cli/README.md.
`marmot message react "$gid" "$id" "🍕"` was producing a kind:7 whose
inner content was four `U+FFFD` replacement characters instead of the
emoji. Whitenoise's `message_aggregator::emoji_utils` rejected it with
`Invalid reaction content: ����`, and the openmls receiver retried the
event ten times before giving up — by which point the secret tree had
already consumed that generation, so the retry surfaced as a confusing
`Ciphertext generation out of bounds 1 / SecretReuseError`. That last
error was a downstream symptom; the root cause is argv decoding.
Why it happens: JEP 400 (Java 18+) pins `file.encoding` to UTF-8 but
deliberately leaves `sun.jnu.encoding` — the encoding the JVM uses to
decode argv, filenames, and native interop strings — tied to the OS
locale, and `sun.jnu.encoding` is read **before** the JVM parses any
`-D` flag, so it can't be overridden via `applicationDefaultJvmArgs`.
Under POSIX/C locales (no `LANG`/`LC_ALL` set, common on CI runners
and stripped-down containers) `sun.jnu.encoding` ends up at
`ANSI_X3.4-1968` — i.e. ASCII — and every byte > 0x7F in argv lands
as `U+FFFD` before our code ever sees it. The four UTF-8 bytes of 🍕
(F0 9F 8D 95) become four replacement chars, get signed into the kind:7
content, and the test fails on receipt.
The only place we can set the encoding from is the launching shell.
Patch the gradle-generated start scripts (POSIX `amy`, Windows
`amy.bat`) right after `:cli:startScripts`:
* POSIX: if neither `LANG` nor `LC_ALL` is set, `export LANG=C.UTF-8`
before invoking Java. Existing locale settings are respected.
* Windows: `chcp 65001` to switch the console to UTF-8 before Java
runs.
Tested manually: with `LANG=` (sandbox default) `amy` previously sent
`argv[0]=????` for an emoji argument; with the patch it sends `🍕` and
`sun.jnu.encoding=UTF-8`, and whitenoise accepts the kind:7 reaction.
The interop test 09 polling also needed a fix unrelated to the
encoding bug: wn aggregates kind:7 reactions onto the anchor message
under `.reactions.by_emoji.<emoji>` instead of surfacing them as
standalone messages whose `.content` is the emoji, so the previous
`wait_for_message B "$mls_gid" "🍕"` would have failed even with a
correctly-decoded reaction. The new poll inspects each message's
`.reactions.by_emoji` keys for the emoji literal.
Marmot interop score: 15/16 → **16/16** 🎉https://claude.ai/code/session_013VYkpz8P1mPh9Ejxy9anhJ
Every event Amy observes — drained from a relay subscription,
unwrapped from a NIP-59 gift wrap, or generated locally for publish —
now flows through Context.verifyAndStore: NIP-01 id + signature check
via Event.verify(), then store.insert(). Bad events are dropped with
a stderr log and never reach command code; persistence failures are
logged but do not propagate, so a broken store never breaks a relay
subscription.
- Context.drain() now persists every received event before surfacing
it to the caller. Existing callers (FeedCommand, DmCommands,
ProfileCommands, Marmot sync) get caching for free.
- Context.publish() persists outbound events too, so the local store
reflects what Amy has done even when every relay rejects.
- Three cache-first read helpers expose the most common lookups
without hitting relays: profileOf(pubKey) → kind:0,
relaysOf(pubKey) → kind:10002, contactsOf(pubKey) → kind:3.
- Class doc on Context spells out the contract; README.md gets a new
"Local event store — the source of truth" section pointing at the
on-disk layout and the design plans.
When wn admin-removes A in interop test 14, A processed the proposal
locally — `tree.removeLeaf(A.leafIndex)` blanked her leaf and shrank
`tree.leafCount` past `myLeafIndex` — and then immediately walked into
BinaryTree.directPath(myLeafIndex, tree.leafCount)
at MlsGroup.processCommitInner. With `myLeafIndex >= leafCount`, the
left-balanced parent walk had no valid stopping point: each recursion
step doubled the candidate parent index, integer-overflowed past 2^31,
and either returned garbage or kept appending to the result list until
the JVM OOM'd. The OOM bubbled up as a `runBlocking` failure that
rolled back the whole commit — so A also stayed locally convinced she
was still a member, and the test timed out waiting for `not_member`.
Three layered fixes so the failure mode can't recur:
* `BinaryTree.parent`/`directPath`/`copath` now `require` an in-range
input. The root and any node ≥ nodeCount used to silently loop or
return garbage; now they throw `IllegalArgumentException` immediately.
This is defense-in-depth — a future caller passing an invalid index
gets a stack trace at the boundary instead of an OOM five frames
deep.
* `MlsGroup.processCommitInner` short-circuits the path-decrypt + epoch
advancement when the proposals in the commit just removed *us*. We
preserve the proposal-side tree mutations so the caller (and the
outer state machine) can observe that we're out, clear pending
proposals + sent keys, and return. Without this short-circuit the
function would derive a bogus all-zero `commit_secret`, fail
`confirmation_tag` verification, throw, and roll the snapshot back —
leaving us still convinced we were a live member.
* `MlsGroupManager.isMember` and a new `MlsGroup.isLocalMember()`
helper now return false once our leaf is null or past `leafCount`.
The post-Remove group entry still lives in `groups` so callers can
inspect the final tree, but every cli command (`group show`,
`message send`, etc.) sees `not_member` and returns the right error.
Also add a comprehensive `BinaryTreeTest` covering non-power-of-2
trees (3, 5, …, 32 leaves) and the boundary cases (root has no
parent, leafIndex ≥ leafCount must throw). The pre-existing tests
only exercised the 4-leaf example from RFC 9420 Appendix C; nothing
hit the `parentInRange` branch, which is exactly where the OOM lived.
Test 14's bash polling needed a small companion fix: it captured
amy's stderr through `2>&1` and fed the result to `jq`, but the
captured stream is interleaved with quartz `Log.d(…)` debug lines,
so the JSON parse always failed. Switch to a `grep` for the
`"error":"not_member"` literal — that signature only appears in
the JSON payload and survives the debug noise. Also tee the
captured output into `$LOG_FILE` so post-mortem logs include each
polling iteration's `[cli] ingest …` traces.
Marmot interop score: 11/16 → 14/16 (tests 9, 15 are unrelated MLS
issues — see follow-up).
https://claude.ai/code/session_013VYkpz8P1mPh9Ejxy9anhJ
- DataDir.eventsDir → "<root>/events-store" (avoids colliding with the
marmot file name conventions).
- Context.store: lazy IEventStore over that directory. Lazy so the
many existing commands that don't touch persistent event state pay
zero open cost (no .lock, no seed file). Closed by Context.close()
only when the lazy delegate has actually been initialised — checked
via reflection so close() never accidentally forces it.
Whitenoise-rs ≥ v0.2.x nests the group object one level deeper —
`{"result": {"group": {...}}}` instead of `{"result": {...}}`. The two
metadata-name pollers in tests 07 and 10 still asked for `.result.name`
and silently saw the empty string, so even when wn-side `groups show`
returned the correct new name the polling loops timed out:
07 metadata rename fail B saw name="" not "Interop-02-renamed"
10 concurrent commits fail diverged: A sees "race-from-amethyst", B sees ""
Both queries now also peel a `.group` wrapper when present, leaving the
older bare-`result` shape working too. Re-running the headless harness
flips 07 + 10 from fail → pass; the remaining failures (09, 14, 15) are
unrelated MLS-encryption / Remove-commit issues that need their own fix.
https://claude.ai/code/session_013VYkpz8P1mPh9Ejxy9anhJ
dm-03 and dm-04 spawn a throwaway "ghost" identity by calling $AMY_BIN
directly (not through amy_a / amy_d), so they missed the plaintext
backend flag added in the previous commit and were failing with
"No TTY and no passphrase source" on headless CI.
https://claude.ai/code/session_01SqdMfLdXvb3GskFLcEj739
identity.json previously stored `privKeyHex` and `nsec` as plaintext fields.
0600 file perms keep other OS users out, but not another app running as the
same user — and that is the threat model the CLI actually cares about.
Introduce a SecretStore indirection:
* identity.json now persists only the public parts plus a typed
`secret: IdentitySecret` envelope (keychain | ncryptsec | plaintext).
* macOS uses `/usr/bin/security` (Keychain ACLs bind the item to the binary
that stored it, so other same-user apps need user consent).
* Linux uses `secret-tool` if a Secret Service is running on the session
D-Bus; the gain there is at-rest encryption while the keyring is locked.
* On any platform without a keychain, auto falls back to NIP-49
(scrypt + XChaCha20) with the passphrase read from --passphrase-file,
then $AMY_PASSPHRASE, then a TTY prompt. Another same-user app can read
the blob but cannot decrypt it without the passphrase.
* `--secret-backend=plaintext` is an explicit opt-in for dev scripts and
the interop test harness.
Legacy identity.json files that still carry top-level privKeyHex/nsec are
read transparently and auto-migrate on the next save.
whoami, `create` / `login` existence checks, and init-re-run now use a
metadata-only load path so they do not trigger a keychain prompt or ask
for a passphrase just to echo the npub.
Tests: cli/tests/ setups wire --secret-backend=plaintext through the amy_a
/ amy_d wrappers so headless CI runs do not stall on a TTY passphrase
prompt.
https://claude.ai/code/session_01SqdMfLdXvb3GskFLcEj739
Three-part design doc for an IEventStore backed by a directory tree
instead of SQLite, targeted at the cli/ module.
- overview: goals, directory layout, feature-parity matrix, public API
- pipelines: insert (T1-T8 with crash-safety), query planner, delete,
transactions, concurrency
- nips: replaceable/addressable slot enforcement via hardlinks +
atomic rename, NIP-09 tombstones (hardlink to the kind-5), NIP-40
expirations, NIP-50 FTS via inverted-index hardlinks, NIP-62 vanish
cascade, tag indexing, seed file, scrub/compact, test strategy,
10-step rollout
Adds a new build-cli matrix to create-release.yml so every tag push also
produces self-contained amy binaries:
- amy-<ver>-macos-x64.tar.gz (macos-13 runner)
- amy-<ver>-macos-arm64.tar.gz (macos-14 runner)
- amy-<ver>-linux-x64.tar.gz (ubuntu-latest)
- amy-<ver>-linux-x64.deb (ubuntu-latest)
- amy-<ver>-linux-x64.rpm (ubuntu-latest)
Each tarball is a flat tree (bin/amy + lib/*.jar + runtime/) with a
jlink'd JDK 21 embedded — no system Java required on the user machine.
The .deb / .rpm install the same tree under /opt/amy/.
Windows is intentionally deferred until cli/ is validated on Windows
(data-dir path handling, file locking on groups/<gid>.mls, identity.json
line endings).
Two follow-ups flagged in comments:
- :commons leaks Compose + Skiko (~40 MB) as transitive deps to :cli.
Tarball lands at ~98 MB instead of the plan's <80 MB target. Size
budget in the workflow is set to 200 MB until :commons is split into
core + ui modules.
- .deb/.rpm install to /opt/amy/bin/amy with no /usr/local/bin symlink.
Users must add to PATH or symlink manually after install.
Wire-up:
- cli/build.gradle.kts gains jlinkRuntime + amyImage (Sync task that
combines installDist output with the jlink runtime + a shell
launcher) + jpackageDeb + jpackageRpm.
- scripts/asset-name.sh gains cli_asset_name() + collect_cli_assets()
alongside the existing desktop collector, preserving the
<product>-<version>-<family>-<arch>.<ext> naming contract.
See cli/plans/2026-04-21-cli-distribution.md for the overall plan.
https://claude.ai/code/session_01Tbh6F7TtEeceb4K3stcUWp
identity.json, relays.json, state.json, MLS group state/retained epochs,
keypackage bundles, and the Marmot message archive were written with
default umask — typically world-readable on Unix. On a shared machine
another OS user could read the stored private key or decrypted chat
history.
Introduce SecureFileIO: overwrites go via a sibling tempfile with POSIX
0600 attrs set at creation time and ATOMIC_MOVE into place, directories
are created 0700, and on upgrade DataDir.init tightens any pre-existing
loose files. Non-POSIX filesystems (Windows) fall back to setReadable/
Writable/Executable to strip other-user access.
This is defense against other OS users. Another app running as the same
user is still in scope — the FileStores doc continues to note that real
deployments MUST encrypt at rest.
https://claude.ai/code/session_01SqdMfLdXvb3GskFLcEj739
Two on-disk layout changes that only affect freshly-created data-dirs (no
migration shim since `amethyst-cli-data` was never tagged or released):
- Default `--data-dir` is now `./amy` instead of `./amethyst-cli-data` —
shorter, matches the binary name, and still overridable via the flag or
`$AMETHYST_CLI_DATA`.
- MLS-only files (`groups/` and `keypackages.bundle`) move under a new
`marmot/` subdirectory so the top-level root stays tidy as more
non-Marmot state lands (notes, profile caches, etc). Existing
top-level files (`identity.json`, `relays.json`, `state.json`) are
unchanged.
README tree diagram and DEVELOPMENT.md test table updated to match.
Moves the top-level `post` and `feed` verbs under a single `notes` group
(`amy notes post`, `amy notes feed`). Keeps the per-event-family grouping
consistent with `profile` (kind:0), `dm` (NIP-17), and `marmot` (MLS).
PostCommand and FeedCommand are unchanged — the new NotesCommands object
is a thin dispatcher that routes `notes post` / `notes feed` into them.
Adds three new top-level verbs to amy that mirror the core social-feed
surface of the Android client:
- `amy profile show [USER]` / `amy profile edit ...` reads and patches the
user's NIP-01 kind:0 metadata. `edit` reuses MetadataEvent.updateFromPast
semantics so unset flags keep prior values and blank values delete the
field, falling back to MetadataEvent.createNew when no kind:0 exists yet.
- `amy post TEXT` publishes a NIP-10 kind:1 short text note to the user's
outbox relays via TextNoteEvent.build.
- `amy feed [--author USER] [--following] [--limit N] [--since|--until TS]`
reads kind:1 notes — own / single-author / contact-list — using
Context.drain with author-scoped filters, dedups by id, and returns the
newest-first window as JSON.
All three keep the thin-assembly-layer rule: no Nostr or business logic in
cli/, every event is built/signed via quartz/ and published through the
existing Context.publish + Context.drain helpers.
Delete() returning false on a still-existing file meant we silently lost
scratch state for MLS group / key-package / message stores. Route the
four call sites through a `deleteOrWarn` helper that logs via the
KMP-safe quartz Log when the file refuses to disappear.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After the harness move from tools/marmot-interop/ to cli/tests/,
three doc locations were still describing the old layout:
- cli/DEVELOPMENT.md § Testing — the "round-trip" row pointed at a
non-existent `cli/src/test/resources/scripts/`, and the "interop
with other clients" row claimed it was out of scope. Both now name
`cli/tests/marmot/` and `cli/tests/dm/` with what each covers, and
the interop-script template points at the concrete test files
rather than re-inventing a smaller example.
- cli/README.md — the "For an interop-test script template" pointer
now links to `cli/tests/README.md` alongside the DEVELOPMENT.md
section.
- .claude/skills/amy-expert/SKILL.md — the "Where things live" tree
omitted the new `tests/` subtree entirely. Added it with a
one-line description per suite. Also extended the wire-up checklist
with a step 7: add a harness case when a new verb changes observable
wire behaviour.
No content changes elsewhere; just path corrections.
Two fixes needed to make the DM harness actually pass end-to-end:
1. Bind the loopback relay to 127.0.0.2 instead of 127.0.0.1.
Quartz's `RelayTag.parse` (called from `ChatMessageRelayListEvent.
relays()`) rejects any URL for which `isLocalHost()` returns true —
the substring check matches `"127.0.0.1"` among others. That means a
kind:10050 event whose `relay` tag reads `ws://127.0.0.1:8090/`
silently parses to an empty relay list, and `RecipientRelayFetcher`
reports `dmInbox == []`. Under strict mode `dm send` then correctly
refuses with `no_dm_relays` — hiding an otherwise-valid test setup.
`127.0.0.2` is the cleanest fix: still pure loopback (no network
traffic, no config needed), not matched by `isLocalHost()`, and
Linux binds any IP in 127.0.0.0/8 without setup. The Marmot harness
is unaffected because its tests never depend on parsing their own
kind:10051/10002 relay tags back out — they hit the bootstrap
fallback path.
marmot/setup.sh's `start_local_relay` now reads `${RELAY_HOST:-
127.0.0.1}` so both harnesses can share it; the DM harness sets
`RELAY_HOST=127.0.0.2` by default and exposes `--host` as an escape
hatch.
2. Replace dm-06 cursor-advance check with `--since` filter check.
The original assertion ("second no-flag `dm list` returns 0") was
wrong for the actual design: `filterGiftWrapsToPubkey` always
subtracts a 2-day lookback from `since` (required to catch NIP-59's
randomised `created_at`), so a repeat no-flag query legitimately
re-pulls everything in the last two days. The cursor advances the
scan window for efficiency, not for idempotency. dm-06 now verifies
`--since` actually filters: query with `--since <newest + 2 days +
1 hour>` — after the filter's lookback subtraction, the effective
floor lands past every message, and the list comes back empty.
Also: added `protoc` to the DM preflight checks so the missing-dep
error comes before 4 failed cargo retries.
Result on this machine: 6/6 pass, two clean runs in a row.
Adds two inner-event verbs on top of the existing kind:9 text send path:
amy marmot message react GID TARGET_EVENT_ID EMOJI
amy marmot message delete GID TARGET_EVENT_ID...
Both build an unsigned rumor per MIP-03 (RumorAssembler) and wrap it in
the usual kind:445 outer via MarmotOutboundProcessor — the same shape as
text messages, just a different inner kind. The target event is looked
up in the account's decrypted inner-message log (persisted by the
inbound pipeline) so the NIP-25 / NIP-09 templates can populate the
target's pubkey and kind tags without making the caller re-derive them.
Test 09 previously noted that react round-trip verification was blocked
on a missing CLI verb. Expanded it to react via amy and confirm B sees
the kind:7 surface in its messages stream.
https://claude.ai/code/session_01Unm6uLHGLj9UcBY7hWfJVW
`tools/` is for libraries (arti-build); shell-based interop harnesses
that drive the `amy` binary belong with the CLI module. New layout:
cli/tests/
├── lib.sh # shared logging + result tracking
├── headless/helpers.sh # shared amy_a / amy_json / assertions
├── marmot/ # MLS group-messaging interop (vs whitenoise-rs)
│ ├── marmot-interop.sh
│ ├── marmot-interop-headless.sh
│ ├── setup.sh
│ ├── tests-create.sh
│ ├── tests-manage.sh
│ ├── tests-extras.sh
│ └── patches/
└── dm/ # NIP-17 DM interop (amy ↔ amy)
├── dm-interop-headless.sh
├── setup.sh
└── tests-dm.sh
Per-suite changes:
- Each suite owns its own setup.sh; previously the Marmot setup was at
headless/setup.sh and the DM setup at headless/setup-dm.sh, which
implied shared infra they don't actually share.
- `cli/tests/lib.sh` and `cli/tests/headless/helpers.sh` are the only
shared bits across suites.
- The DM harness still reuses Marmot's `start_local_relay` /
`stop_local_relay` (sourced via `../marmot/setup.sh`) — same relay
binary, no duplication.
- All sourcing paths updated; REPO_ROOT now climbs three levels (was
two when the harness lived under `tools/`); patches/ is now a sibling
of marmot/setup.sh rather than under marmot/headless/.
- `.gitignore` updated to cover both suites' state dirs.
cli/ROADMAP.md and cli/tests/README.md updated to point at the new paths.
No behavioural change — every test runs the same code; only the file
layout and source paths moved.
Mirrors the Android Danger-Zone action: wipes every MLS group, retained
epoch secret, persisted KeyPackage bundle, relay subscription, and the
run-state since-cursors. Does not broadcast any SelfRemove/leave
commits, because the reset path is for recovering from corrupted or
unrecoverable local state where graceful teardown may be impossible.
Requires an explicit --yes to execute. Without --yes it emits a dry-run
JSON listing the groups that would be wiped and exits 2, so scripts
that forget the flag fail loudly instead of nuking state.
https://claude.ai/code/session_01Unm6uLHGLj9UcBY7hWfJVW
Adds `tools/marmot-interop/dm-interop-headless.sh` — a zero-prompt
end-to-end harness that runs two independent `amy` processes against
a loopback nostr-rs-relay and verifies the NIP-17 DM surface.
Six scenarios:
dm-01 text round-trip (kind:14) in both directions
dm-02 `dm list` surfaces prior exchange with `type:text` discriminator
dm-03 strict kind:10050 refuses sends to an inboxless recipient
(surfaces the `no_dm_relays` error JSON)
dm-04 `--allow-fallback` opts into NIP-65 read / bootstrap chain
(asserts a `relay_source: "bootstrap"` or `"nip65_read"` row)
dm-05 file message reference mode (kind:15) — send-file with
--key / --nonce, verify recipient decodes url + key + nonce + mime
dm-06 no-flag `dm list` advances `state.giftWrapSince`; second call
returns an empty `messages` array
Shape matches the existing Marmot harness: reuses `lib.sh` for logging
+ result tracking, reuses `setup.sh` for the nostr-rs-relay lifecycle
(`start_local_relay` / `stop_local_relay`), adds a slim `setup-dm.sh`
preflight (amy + relay only, no whitenoise-rs / Marmot patches), and
defines tests in `tests-dm.sh`.
No new CI job yet — the relay build needs Rust + ~3 minutes on a
cold cache, same constraint as the Marmot harness.
Upload-mode (`dm send-file --file PATH --server URL`) is not scripted
here: it needs a local Blossom server, which is out of scope for this
pass. The shared upload classes are unit-tested on desktop at
`desktopApp/src/jvmTest/kotlin/.../service/upload/`.
`dm send-file` now has two modes:
- **Upload mode** — `dm send-file RECIPIENT --file PATH --server URL`:
generate a fresh AES-GCM cipher, encrypt the file, upload the
ciphertext to the Blossom server, then publish a kind:15 referencing
the returned URL. Auto-detected metadata (encrypted hash + size,
original sha256, mime type, image dimensions, blurhash) is folded
into the kind:15 tags. The response surfaces the encryption key and
nonce so the same encrypted blob can be re-shared without
re-uploading.
- **Reference mode** — the previous shape still works:
`dm send-file RECIPIENT URL --key HEX --nonce HEX [...]`. Useful
when the upload happened elsewhere (Android client, third-party
tool) or for replaying a previous upload.
Mode is selected by the presence of `--file`. Both paths share the
same wrap-and-publish pipeline (`publishWraps`) and respect the strict
NIP-17 §6 relay rule: kind:1059 only reaches the recipient's kind:10050
unless `--allow-fallback` is passed.
Implementation entirely reuses the upload helpers just promoted to
`commons/jvmMain/.../service/upload/`, so no new business logic lives
in `cli/`. The Blossom auth event (kind:24242) is built and signed via
`BlossomAuth.createUploadAuth(... ctx.signer ...)`.
Receive:
- decryptDms now accepts both ChatMessageEvent (kind:14) and
ChatMessageEncryptedFileHeaderEvent (kind:15). The result is a sealed
DecryptedDm hierarchy (TextDm / FileDm) with a `type: "text"|"file"`
discriminator in JSON. File messages emit url, mime_type,
encryption_algorithm, decryption_key, decryption_nonce, hash,
original_hash, size, dim, blurhash. `dm await --match X` searches the
text body for kind:14 and the URL for kind:15.
Send:
- New `dm send-file RECIPIENT URL --key HEX --nonce HEX [...]` verb.
Builds a kind:15 ChatMessageEncryptedFileHeaderEvent via the existing
ChatMessageEncryptedFileHeaderEvent.build + AESGCM(key, nonce), then
publishes through NIP17Factory.createEncryptedFileNIP17. The file is
expected to already be uploaded; carrying the upload itself in `amy`
is the next change.
Spec compliance (NIP-17 §6, "if no kind:10050, shouldn't try"):
- resolveDmRelays now returns kind:10050 only by default. If the
recipient has none, send/send-file exit with `no_dm_relays` and a
message recommending `--allow-fallback`. The fallback chain (kind:10002
read marker → bootstrap) is preserved behind that opt-in flag for
interop tests and brand-new accounts.
Each recipient row in the send response now includes `relay_source`
("kind_10050" / "nip65_read" / "bootstrap") so callers can tell where
the wraps actually went.
Note: Quartz already enforces the NIP-17 §7 step-3 impersonation guard
(seal pubkey is forced over the rumor's claimed pubkey inside
`Rumor.mergeWith`), so no extra check is needed in the CLI.
davotoula