fix(marmot): trim trailing blank leaves after removeLeaf

RFC 9420 §7.8: the ratchet tree has no trailing blank leaves — when a
Remove blanks the rightmost leaf, everyone MUST shrink `leaf_count` and
drop the (now-orphaned) parent nodes so future `direct_path`,
`resolution`, and `treeHash` computations see the new shape.

Quartz was blanking the leaf and its direct path but leaving
`_leafCount` untouched. Openmls / mdk shrink, so after
e.g. `amy removes C (leaf 2)` quartz still had `leafCount = 3` while
B's openmls had `leafCount = 2`. Amy's `direct_path` from leaf 0 then
had two entries (to the 3-leaf-tree root), but B's tree layout expected
one — and openmls rejected the commit with
`UpdatePathError(PathLengthMismatch)`.

Fix: after blanking the leaf + direct path, walk `leafCount` down past
any trailing blanks and truncate the `nodes` list to `nodeCount(leafCount)`.

https://claude.ai/code/session_016kAxdp6ubB5CnF9URhCEzP
This commit is contained in:
Claude
2026-04-22 03:28:32 +00:00
parent c50fd18cb5
commit 8313c4a584
@@ -156,16 +156,33 @@ class RatchetTree(
/**
* Remove a member by blanking their leaf and all parent nodes on the direct path.
*
* RFC 9420 §7.8: trailing blank leaves MUST be trimmed so every participant
* agrees on `leaf_count` (and therefore on `direct_path` lengths). Openmls
* does this after every leaf blank; without the trim, a committer that had
* the removed leaf at the far right end of the tree sends an UpdatePath one
* entry longer than the receiver's tree layout accepts, and the receiver
* errors out with `UpdatePathError(PathLengthMismatch)`.
*/
fun removeLeaf(leafIndex: Int) {
setLeaf(leafIndex, null)
// Blank the direct path
val directPath = BinaryTree.directPath(leafIndex, _leafCount)
for (nodeIdx in directPath) {
if (nodeIdx < nodes.size) {
nodes[nodeIdx] = null
}
}
// Shrink leafCount past any trailing blanks. Parent nodes that become
// orphaned by the shrink are dropped from the `nodes` list so
// treeHash() / resolution() / direct_path() agree with openmls on the
// new tree shape.
while (_leafCount > 0 && getLeaf(_leafCount - 1) == null) {
_leafCount--
}
val effectiveNodeCount = if (_leafCount > 0) BinaryTree.nodeCount(_leafCount) else 0
while (nodes.size > effectiveNodeCount) {
nodes.removeAt(nodes.size - 1)
}
}
/**