Files
amethyst/cli/build.gradle.kts
Claude fc99bed55a fix(cli): default amy launcher to C.UTF-8 so emoji argv survives
`marmot message react "$gid" "$id" "🍕"` was producing a kind:7 whose
inner content was four `U+FFFD` replacement characters instead of the
emoji. Whitenoise's `message_aggregator::emoji_utils` rejected it with
`Invalid reaction content: ����`, and the openmls receiver retried the
event ten times before giving up — by which point the secret tree had
already consumed that generation, so the retry surfaced as a confusing
`Ciphertext generation out of bounds 1 / SecretReuseError`. That last
error was a downstream symptom; the root cause is argv decoding.

Why it happens: JEP 400 (Java 18+) pins `file.encoding` to UTF-8 but
deliberately leaves `sun.jnu.encoding` — the encoding the JVM uses to
decode argv, filenames, and native interop strings — tied to the OS
locale, and `sun.jnu.encoding` is read **before** the JVM parses any
`-D` flag, so it can't be overridden via `applicationDefaultJvmArgs`.
Under POSIX/C locales (no `LANG`/`LC_ALL` set, common on CI runners
and stripped-down containers) `sun.jnu.encoding` ends up at
`ANSI_X3.4-1968` — i.e. ASCII — and every byte > 0x7F in argv lands
as `U+FFFD` before our code ever sees it. The four UTF-8 bytes of 🍕
(F0 9F 8D 95) become four replacement chars, get signed into the kind:7
content, and the test fails on receipt.

The only place we can set the encoding from is the launching shell.
Patch the gradle-generated start scripts (POSIX `amy`, Windows
`amy.bat`) right after `:cli:startScripts`:

* POSIX: if neither `LANG` nor `LC_ALL` is set, `export LANG=C.UTF-8`
  before invoking Java. Existing locale settings are respected.
* Windows: `chcp 65001` to switch the console to UTF-8 before Java
  runs.

Tested manually: with `LANG=` (sandbox default) `amy` previously sent
`argv[0]=????` for an emoji argument; with the patch it sends `🍕` and
`sun.jnu.encoding=UTF-8`, and whitenoise accepts the kind:7 reaction.

The interop test 09 polling also needed a fix unrelated to the
encoding bug: wn aggregates kind:7 reactions onto the anchor message
under `.reactions.by_emoji.<emoji>` instead of surfacing them as
standalone messages whose `.content` is the emoji, so the previous
`wait_for_message B "$mls_gid" "🍕"` would have failed even with a
correctly-decoded reaction. The new poll inspects each message's
`.reactions.by_emoji` keys for the emoji literal.

Marmot interop score: 15/16 → **16/16** 🎉

https://claude.ai/code/session_013VYkpz8P1mPh9Ejxy9anhJ
2026-04-25 02:53:38 +00:00

299 lines
11 KiB
Kotlin

import org.jetbrains.kotlin.gradle.dsl.JvmTarget
plugins {
alias(libs.plugins.jetbrainsKotlinJvm)
application
}
kotlin {
jvmToolchain(21)
compilerOptions {
jvmTarget.set(JvmTarget.JVM_21)
}
}
sourceSets {
main {
kotlin.srcDir("src/main/kotlin")
resources.srcDir("src/main/resources")
}
}
dependencies {
implementation(project(":quartz"))
implementation(project(":commons"))
implementation(libs.kotlinx.coroutines.core)
implementation(libs.okhttp)
implementation(libs.okhttpCoroutines)
implementation(libs.jackson.module.kotlin)
implementation(libs.slf4j.nop)
}
application {
mainClass.set("com.vitorpamplona.amethyst.cli.MainKt")
applicationName = "amy"
}
// Inject `LANG=C.UTF-8` (and the matching Windows code page) into the
// launcher scripts when `LANG`/`LC_ALL` aren't already set. Without this,
// running amy under a POSIX/C locale leaves `sun.jnu.encoding` as
// `ANSI_X3.4-1968` (i.e. ASCII) — JEP 400 pins `file.encoding` to UTF-8
// but deliberately leaves `sun.jnu.encoding` tied to the OS locale, and
// it's read by the JVM *before* any `-D` flag has a chance to apply. So
// `-Dsun.jnu.encoding=UTF-8` does nothing; the encoding has to be set in
// the environment that launches Java.
//
// The symptom this fixes: `marmot message react "$gid" "$id" "🍕"` —
// the shell hands the four UTF-8 bytes (F0 9F 8D 95) to the JVM, the
// JVM decodes each one as ASCII (every byte > 0x7F → U+FFFD), and amy
// then signs a kind:7 whose `content` is four replacement characters.
// Whitenoise rejects it with "Invalid reaction content".
val patchAmyLauncherCharset by tasks.registering {
val appName = application.applicationName
val startScriptsTask = tasks.named("startScripts")
dependsOn(startScriptsTask)
// Patch the scripts at their source location so installDist (and any
// downstream packaging like jpackage) copies the patched copies.
doLast {
val unixScript = layout.buildDirectory.file("scripts/$appName").get().asFile
if (unixScript.exists()) {
val text = unixScript.readText()
val marker = "# Default LANG to UTF-8 so the JVM picks UTF-8 for sun.jnu.encoding"
if (!text.contains(marker)) {
// Build the snippet with explicit literal `$` characters to avoid
// regex-replacement-string escapes when we splice it in.
val injected = buildString {
append("\n")
append(marker).append("\n")
append("# (POSIX/C locales force ANSI_X3.4-1968, which mangles non-ASCII argv).\n")
append("if [ -z \"")
append("$").append("{LANG-}").append("$").append("{LC_ALL-}")
append("\" ]; then\n")
append(" export LANG=C.UTF-8\n")
append("fi\n")
}
// Insert right after the shebang. Use indexOf+substring instead
// of regex replaceFirst so the `$` chars in `injected` aren't
// mistaken for backreferences.
val nl = text.indexOf('\n')
val patched =
if (nl >= 0 && text.startsWith("#!")) {
text.substring(0, nl + 1) + injected + text.substring(nl + 1)
} else {
injected.trimStart('\n') + text
}
unixScript.writeText(patched)
}
}
val batScript = layout.buildDirectory.file("scripts/$appName.bat").get().asFile
if (batScript.exists()) {
val text = batScript.readText()
val marker = "rem Pin code page to UTF-8 so sun.jnu.encoding picks UTF-8"
if (!text.contains(marker)) {
val injected =
"$marker\r\n" +
"chcp 65001 > NUL 2>&1\r\n"
val anchor = "@if \"%DEBUG%\"==\"\" @echo off"
val idx = text.indexOf(anchor)
if (idx >= 0) {
val afterAnchor = text.indexOf('\n', idx)
if (afterAnchor >= 0) {
val patched =
text.substring(0, afterAnchor + 1) + injected + text.substring(afterAnchor + 1)
batScript.writeText(patched)
}
}
}
}
}
}
tasks.named("installDist") {
dependsOn(patchAmyLauncherCharset)
}
// ---------------------------------------------------------------------------
// Native distribution (jlink + jpackage)
//
// Produces a self-contained `amy` bundle with a minimal jlink'd JRE embedded —
// no JDK required on the user machine. Outputs land under cli/build/:
// - amy-image/amy/ portable, flat directory (bin/ + lib/ + runtime/)
// tar this up on every OS → amy-<ver>-<fam>-<arch>.tar.gz
// - jpackage/amy_*.deb Debian/Ubuntu package (Linux runners only)
// - jpackage/amy-*.rpm Fedora/RHEL package (Linux runners only)
//
// We deliberately build our own app-image instead of using `jpackage --type
// app-image`, because on macOS jpackage produces an `.app` bundle (with the
// binary buried at Contents/MacOS/amy) — awful UX for a CLI. The flat tree we
// build matches the Linux jpackage layout on every OS.
//
// Packaging for release is wired in .github/workflows/create-release.yml.
// See cli/plans/2026-04-21-cli-distribution.md for the overall plan.
// ---------------------------------------------------------------------------
val appVersion: String = project.version.toString()
// RPM rejects dashes in version strings — replace with tilde (~), which RPM
// treats as prerelease-lower-than: 1.08.0~rc1 < 1.08.0.
val rpmVersion: String = appVersion.replace("-", "~")
val mainJarName: String = "cli-$appVersion.jar"
val mainClassName: String = "com.vitorpamplona.amethyst.cli.MainKt"
// Minimal JDK 21 module set for amy. Keep this tight — every module adds
// megabytes to the bundle. If a transitive dep needs more, `jlink` fails loudly
// at build time with "module X not found".
val jlinkModules: String = listOf(
"java.base",
"java.logging",
"java.naming",
"java.net.http",
"java.sql",
"java.xml",
"jdk.crypto.ec",
"jdk.unsupported",
).joinToString(",")
fun javaToolBin(name: String): Provider<String> =
javaToolchains.launcherFor {
languageVersion.set(JavaLanguageVersion.of(21))
}.map {
val exe = if (org.gradle.internal.os.OperatingSystem.current().isWindows) "$name.exe" else name
it.metadata.installationPath.file("bin/$exe").asFile.absolutePath
}
val jlinkRuntimeDir = layout.buildDirectory.dir("jlink-runtime")
val amyImageRoot = layout.buildDirectory.dir("amy-image")
val amyImageDir = layout.buildDirectory.dir("amy-image/amy")
val installLibDir = layout.buildDirectory.dir("install/amy/lib")
val jpackageOutDir = layout.buildDirectory.dir("jpackage")
val jlinkRuntime =
tasks.register<Exec>("jlinkRuntime") {
group = "distribution"
description = "Build a minimal JRE for amy via jlink."
outputs.dir(jlinkRuntimeDir)
val jlinkBin = javaToolBin("jlink")
val outDir = jlinkRuntimeDir
val modules = jlinkModules
doFirst {
// jlink refuses to write into an existing directory.
outDir.get().asFile.deleteRecursively()
executable = jlinkBin.get()
args(
"--add-modules", modules,
"--no-header-files",
"--no-man-pages",
"--strip-debug",
// JDK 21+: --compress <int> is deprecated; use zip-<level>.
"--compress", "zip-6",
"--output", outDir.get().asFile.absolutePath,
)
}
}
// Flat app-image: bin/amy launcher + lib/*.jar + runtime/ (the jlink'd JRE).
// Cross-platform — the release workflow tars this up on every OS.
val amyImage =
tasks.register<Sync>("amyImage") {
group = "distribution"
description = "Assemble a portable amy app-image (bin/ + lib/ + runtime/)."
dependsOn(tasks.named("installDist"), jlinkRuntime)
into(amyImageDir)
// jars from installDist
from(installLibDir) {
into("lib")
}
// jlink'd JRE
from(jlinkRuntimeDir) {
into("runtime")
}
val mainJar = mainJarName
val mainClass = mainClassName
val unixLauncher =
"""
#!/bin/sh
# amy launcher — uses the bundled jlink'd JRE so no system Java is required.
DIR="${'$'}(cd "${'$'}(dirname "${'$'}0")/.." && pwd)"
exec "${'$'}DIR/runtime/bin/java" -cp "${'$'}DIR/lib/*" $mainClass "${'$'}@"
""".trimIndent() + "\n"
doLast {
val binDir = amyImageDir.get().asFile.resolve("bin")
binDir.mkdirs()
val launcher = binDir.resolve("amy")
launcher.writeText(unixLauncher)
launcher.setExecutable(true, false)
}
}
fun registerJpackage(
taskName: String,
type: String,
extraArgs: List<String> = emptyList(),
) = tasks.register<Exec>(taskName) {
group = "distribution"
description = "Run jpackage --type $type for amy."
dependsOn(tasks.named("installDist"), jlinkRuntime)
inputs.dir(installLibDir)
inputs.dir(jlinkRuntimeDir)
outputs.dir(jpackageOutDir)
val jpackageBin = javaToolBin("jpackage")
val inDir = installLibDir
val runtimeDir = jlinkRuntimeDir
val outDir = jpackageOutDir
val versionArg = if (type == "rpm") rpmVersion else appVersion
val extra = extraArgs
doFirst {
outDir.get().asFile.mkdirs()
executable = jpackageBin.get()
args(
"--type", type,
"--name", "amy",
"--app-version", versionArg,
"--vendor", "Amethyst Contributors",
"--description", "Amethyst CLI — a non-interactive Nostr client.",
"--input", inDir.get().asFile.absolutePath,
"--runtime-image", runtimeDir.get().asFile.absolutePath,
"--main-jar", mainJarName,
"--main-class", mainClassName,
"--dest", outDir.get().asFile.absolutePath,
)
args(extra)
}
}
// .deb for Debian/Ubuntu. Installs under /opt/amy/ with /opt/amy/bin/amy as
// the launcher. We intentionally do NOT request --linux-shortcut (no .desktop
// entry for a CLI).
registerJpackage(
"jpackageDeb",
"deb",
extraArgs = listOf(
"--linux-package-name", "amy",
"--linux-deb-maintainer", "vitor@vitorpamplona.com",
),
)
// .rpm for Fedora/RHEL/openSUSE.
registerJpackage(
"jpackageRpm",
"rpm",
extraArgs = listOf(
"--linux-package-name", "amy",
"--linux-rpm-license-type", "MIT",
),
)