refactor(quartz): tidy projection package, flatten StoreEvent, per-filter limits
Five changes wrapped together:
1. Move ObservableEventStore + StoreEvent from store/observable/ into
store/projection/. One package owns the projection bus + projection
itself; consumers import only `store.projection.*`.
2. Flatten DeleteRule into the StoreEvent sealed type:
StoreEvent.Insert(event)
StoreEvent.DeleteByFilter(filters)
StoreEvent.DeleteExpired(asOf?)
ObservableEventStore.delete(filter|filters) emits DeleteByFilter;
deleteExpiredEvents() emits DeleteExpired with the pinned cutoff.
3. Use event.address() for AddressableEvent and event.isExpirationBefore(t)
for NIP-40 checks; drop the local addressOf(kind, pubKey, dTag) and
isExpiredAt helpers. Plain replaceables (kind 0/3, 10000-19999) still
build Address(kind, pubKey, "") manually since they don't implement
AddressableEvent. NIP-09 delete-by-address now looks up the projection's
byAddress index with the deletion's Address directly (it's already an
Address, no conversion needed).
4. Per-filter limit. Each filter retains its own capped TreeSet<Slot>
(sorted created_at DESC, id ASC). A slot is "live" iff at least one
filter retains it. The projection's items is the deduped union, so
when filter A and filter B match disjoint events the union can
exceed any single filter's limit. New tests confirm both behaviours:
per-filter eviction + union-larger-than-cap.
5. Drop the redundant + 1 in expiration checks; use isExpirationBefore
directly with TimeUtils.now() for arrival-time guards. The sweep
path subtracts 1 because the store's SQL uses strict `<` while the
helper is `<=`.
17/17 projection tests + all other store tests pass.
https://claude.ai/code/session_01Jny85MTu1ynKgFBgysfWu5
This commit is contained in:
+131
-98
@@ -24,15 +24,11 @@ import com.vitorpamplona.quartz.nip01Core.core.Address
|
|||||||
import com.vitorpamplona.quartz.nip01Core.core.AddressableEvent
|
import com.vitorpamplona.quartz.nip01Core.core.AddressableEvent
|
||||||
import com.vitorpamplona.quartz.nip01Core.core.Event
|
import com.vitorpamplona.quartz.nip01Core.core.Event
|
||||||
import com.vitorpamplona.quartz.nip01Core.core.HexKey
|
import com.vitorpamplona.quartz.nip01Core.core.HexKey
|
||||||
import com.vitorpamplona.quartz.nip01Core.core.isAddressable
|
|
||||||
import com.vitorpamplona.quartz.nip01Core.core.isReplaceable
|
import com.vitorpamplona.quartz.nip01Core.core.isReplaceable
|
||||||
import com.vitorpamplona.quartz.nip01Core.relay.filters.Filter
|
import com.vitorpamplona.quartz.nip01Core.relay.filters.Filter
|
||||||
import com.vitorpamplona.quartz.nip01Core.relay.normalizer.NormalizedRelayUrl
|
import com.vitorpamplona.quartz.nip01Core.relay.normalizer.NormalizedRelayUrl
|
||||||
import com.vitorpamplona.quartz.nip01Core.store.observable.DeleteRule
|
|
||||||
import com.vitorpamplona.quartz.nip01Core.store.observable.ObservableEventStore
|
|
||||||
import com.vitorpamplona.quartz.nip01Core.store.observable.StoreEvent
|
|
||||||
import com.vitorpamplona.quartz.nip09Deletions.DeletionEvent
|
import com.vitorpamplona.quartz.nip09Deletions.DeletionEvent
|
||||||
import com.vitorpamplona.quartz.nip40Expiration.expiration
|
import com.vitorpamplona.quartz.nip40Expiration.isExpirationBefore
|
||||||
import com.vitorpamplona.quartz.nip59Giftwrap.wraps.GiftWrapEvent
|
import com.vitorpamplona.quartz.nip59Giftwrap.wraps.GiftWrapEvent
|
||||||
import com.vitorpamplona.quartz.nip62RequestToVanish.RequestToVanishEvent
|
import com.vitorpamplona.quartz.nip62RequestToVanish.RequestToVanishEvent
|
||||||
import com.vitorpamplona.quartz.utils.TimeUtils
|
import com.vitorpamplona.quartz.utils.TimeUtils
|
||||||
@@ -49,9 +45,8 @@ import kotlinx.coroutines.yield
|
|||||||
/**
|
/**
|
||||||
* A reactive projection over an [ObservableEventStore] for a fixed
|
* A reactive projection over an [ObservableEventStore] for a fixed
|
||||||
* set of [filters]. Each visible event is wrapped in a
|
* set of [filters]. Each visible event is wrapped in a
|
||||||
* [MutableStateFlow] so
|
* [MutableStateFlow] so the UI can collect three different kinds of
|
||||||
* the UI can collect three different kinds of change with the right
|
* change with the right granularity:
|
||||||
* granularity:
|
|
||||||
*
|
*
|
||||||
* - **Membership** (events arriving or leaving) re-emits a brand new
|
* - **Membership** (events arriving or leaving) re-emits a brand new
|
||||||
* [List] from [items]. The list reference is stable while membership
|
* [List] from [items]. The list reference is stable while membership
|
||||||
@@ -63,17 +58,16 @@ import kotlinx.coroutines.yield
|
|||||||
* reshuffled when the new version has a later `created_at` — each
|
* reshuffled when the new version has a later `created_at` — each
|
||||||
* slot remembers the sort key it was inserted with, so updates feel
|
* slot remembers the sort key it was inserted with, so updates feel
|
||||||
* like pure value mutations.
|
* like pure value mutations.
|
||||||
* - **Removal** (NIP-09 deletion, NIP-62 vanish, NIP-40 expiration)
|
* - **Removal** (NIP-09 deletion, NIP-62 vanish, NIP-40 expiration,
|
||||||
* drops the handle from the list.
|
* `delete(filter)`) drops the handle from the list.
|
||||||
*
|
*
|
||||||
* The seed is materialised by querying the store once at start,
|
* The seed is materialised by querying the store once at start, after
|
||||||
* after which the projection is driven entirely by
|
* which the projection is driven entirely by
|
||||||
* [ObservableEventStore.events]. Two kinds of mutation arrive on
|
* [ObservableEventStore.events]. Three kinds of mutation arrive on
|
||||||
* that stream:
|
* that stream:
|
||||||
*
|
*
|
||||||
* - [StoreEvent.Insert] — interpreted in-projection so a single
|
* - [StoreEvent.Insert] — interpreted in-projection so a single
|
||||||
* arriving event can carry NIP-01 / NIP-09 / NIP-62 semantics:
|
* arriving event can carry NIP-01 / NIP-09 / NIP-62 semantics:
|
||||||
*
|
|
||||||
* - **NIP-01 supersession.** New replaceable / addressable events
|
* - **NIP-01 supersession.** New replaceable / addressable events
|
||||||
* replace prior ones for the same `kind:pubkey[:dtag]`. The
|
* replace prior ones for the same `kind:pubkey[:dtag]`. The
|
||||||
* NIP-01 lexical-id tiebreaker (`new.id < old.id` when
|
* NIP-01 lexical-id tiebreaker (`new.id < old.id` when
|
||||||
@@ -88,20 +82,23 @@ import kotlinx.coroutines.yield
|
|||||||
* already lapsed at the moment they arrive are dropped before
|
* already lapsed at the moment they arrive are dropped before
|
||||||
* they ever enter [items].
|
* they ever enter [items].
|
||||||
*
|
*
|
||||||
* - [StoreEvent.Delete] — emitted when a caller invokes
|
* - [StoreEvent.DeleteByFilter] — emitted on `delete(filter)` /
|
||||||
* `delete(filter)`, `delete(filters)`, or `deleteExpiredEvents()`
|
* `delete(filters)`. The projection drops every slot matching any
|
||||||
* on the [ObservableEventStore]. The projection drops every slot
|
* of the rule's filters via [Filter.match].
|
||||||
* matching the rule using the same [Filter.match] / NIP-40
|
*
|
||||||
* expiration semantics the store used. **There is no per-projection
|
* - [StoreEvent.DeleteExpired] — emitted on `deleteExpiredEvents()`.
|
||||||
|
* The projection drops every slot whose `expiration` has lapsed at
|
||||||
|
* the cutoff the store pinned. **There is no per-projection
|
||||||
* expiration ticker** — projections only drop expired events when
|
* expiration ticker** — projections only drop expired events when
|
||||||
* the application calls `deleteExpiredEvents()` on the store.
|
* the application calls `deleteExpiredEvents()` on the store.
|
||||||
*
|
*
|
||||||
* Limit handling: the initial query honours the filter `limit`, and
|
* Limit handling is **per-filter**: each filter retains at most its
|
||||||
* we trim to the same cap when an insert pushes the list over. We do
|
* own `limit` matches in a private capped set, sorted by created_at
|
||||||
* **not** refill from the store after a deletion — if a deletion
|
* DESC + id ASC. The projection's [items] is the deduped union of
|
||||||
* leaves you under the limit, you stay under the limit until something
|
* those sets, so when filter A and filter B match disjoint events the
|
||||||
* new arrives. That tradeoff matches what callers were already getting
|
* union can be larger than any single filter's `limit`. We do not
|
||||||
* from `LocalCache.observeEvents`.
|
* refill from the store after a deletion — if a removal leaves a
|
||||||
|
* filter under cap, it stays under cap until another match arrives.
|
||||||
*
|
*
|
||||||
* Ephemeral events (kinds `20000-29999`) reach the projection via
|
* Ephemeral events (kinds `20000-29999`) reach the projection via
|
||||||
* [ObservableEventStore.events] without ever being persisted; they
|
* [ObservableEventStore.events] without ever being persisted; they
|
||||||
@@ -138,13 +135,20 @@ class EventStoreProjection<T : Event>(
|
|||||||
private val byAddress = HashMap<Address, Slot<T>>()
|
private val byAddress = HashMap<Address, Slot<T>>()
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Sorted view of the same slots. The comparator uses each slot's
|
* Per-filter capped sets. Each filter independently retains at most
|
||||||
* frozen sort key (the seed event's `created_at` + `id`), so
|
* `filter.limit` matches; a slot is "live" (visible in [items])
|
||||||
* supersession in-place updates never move a slot inside this set.
|
* iff it appears in at least one of these sets. Identity-keyed —
|
||||||
|
* [Filter] is `@Stable` but not a data class.
|
||||||
*/
|
*/
|
||||||
private val ordered = sortedSetOf(slotComparator<T>())
|
private val perFilter: Map<Filter, java.util.SortedSet<Slot<T>>> =
|
||||||
|
filters.associateWith { sortedSetOf(slotComparator()) }
|
||||||
|
|
||||||
private val limit: Int? = filters.mapNotNull { it.limit }.maxOrNull()
|
/**
|
||||||
|
* Sorted union view of every "live" slot. Maintained alongside
|
||||||
|
* [perFilter] — a slot is added the first time any filter retains
|
||||||
|
* it and removed once no filter still holds it.
|
||||||
|
*/
|
||||||
|
private val ordered: java.util.SortedSet<Slot<T>> = sortedSetOf(slotComparator())
|
||||||
|
|
||||||
/** Set when the seed has been written to [items], so callers can suspend until the projection is hot. */
|
/** Set when the seed has been written to [items], so callers can suspend until the projection is hot. */
|
||||||
val ready: CompletableDeferred<Unit> = CompletableDeferred()
|
val ready: CompletableDeferred<Unit> = CompletableDeferred()
|
||||||
@@ -171,8 +175,8 @@ class EventStoreProjection<T : Event>(
|
|||||||
// do it here — otherwise an expired event would briefly
|
// do it here — otherwise an expired event would briefly
|
||||||
// appear in [items] before the next deleteExpiredEvents()
|
// appear in [items] before the next deleteExpiredEvents()
|
||||||
// sweep clears it.
|
// sweep clears it.
|
||||||
if (isExpiredAt(event, now)) continue
|
if (event.isExpirationBefore(now)) continue
|
||||||
insertNew(event)
|
applyInsert(event)
|
||||||
yield()
|
yield()
|
||||||
}
|
}
|
||||||
publish()
|
publish()
|
||||||
@@ -180,13 +184,23 @@ class EventStoreProjection<T : Event>(
|
|||||||
|
|
||||||
private fun apply(storeEvent: StoreEvent) {
|
private fun apply(storeEvent: StoreEvent) {
|
||||||
when (storeEvent) {
|
when (storeEvent) {
|
||||||
is StoreEvent.Insert -> applyInsert(storeEvent.event)
|
is StoreEvent.Insert -> {
|
||||||
is StoreEvent.Delete -> applyDelete(storeEvent.rule)
|
if (applyInsert(storeEvent.event)) publish()
|
||||||
|
}
|
||||||
|
|
||||||
|
is StoreEvent.DeleteByFilter -> {
|
||||||
|
if (applyDeleteByFilter(storeEvent.filters)) publish()
|
||||||
|
}
|
||||||
|
|
||||||
|
is StoreEvent.DeleteExpired -> {
|
||||||
|
val cutoff = storeEvent.asOf ?: nowProvider()
|
||||||
|
if (applyDeleteExpired(cutoff)) publish()
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private fun applyInsert(event: Event) {
|
private fun applyInsert(event: Event): Boolean {
|
||||||
if (isExpiredAt(event, nowProvider())) return
|
if (event.isExpirationBefore(nowProvider())) return false
|
||||||
|
|
||||||
var changed = false
|
var changed = false
|
||||||
|
|
||||||
@@ -201,49 +215,54 @@ class EventStoreProjection<T : Event>(
|
|||||||
if (handleVanish(event)) changed = true
|
if (handleVanish(event)) changed = true
|
||||||
}
|
}
|
||||||
|
|
||||||
if (filters.any { it.match(event) }) {
|
if (handleInsert(event)) changed = true
|
||||||
if (handleInsert(event)) changed = true
|
|
||||||
}
|
|
||||||
|
|
||||||
if (changed) publish()
|
return changed
|
||||||
}
|
}
|
||||||
|
|
||||||
private fun applyDelete(rule: DeleteRule) {
|
private fun applyDeleteByFilter(rules: List<Filter>): Boolean {
|
||||||
val targets =
|
if (rules.isEmpty()) return false
|
||||||
when (rule) {
|
val targets = byId.values.filter { slot -> rules.any { it.match(slot.flow.value) } }
|
||||||
is DeleteRule.Filtered -> {
|
if (targets.isEmpty()) return false
|
||||||
if (rule.filters.isEmpty()) return
|
|
||||||
byId.values.filter { slot -> rule.filters.any { it.match(slot.flow.value) } }
|
|
||||||
}
|
|
||||||
|
|
||||||
is DeleteRule.Expired -> {
|
|
||||||
val cutoff = rule.asOf ?: nowProvider()
|
|
||||||
byId.values.filter { isExpiredAt(it.flow.value, cutoff) }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (targets.isEmpty()) return
|
|
||||||
var changed = false
|
var changed = false
|
||||||
for (slot in targets) {
|
for (slot in targets) {
|
||||||
if (removeSlot(slot)) changed = true
|
if (removeSlot(slot)) changed = true
|
||||||
}
|
}
|
||||||
if (changed) publish()
|
return changed
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun applyDeleteExpired(asOf: Long): Boolean {
|
||||||
|
// Store's sweep uses strict `<`; isExpirationBefore is `<=`,
|
||||||
|
// so subtract 1 to match. (Resolution is 1 second; the
|
||||||
|
// off-by-one in the rare equal-timestamp case lines up with
|
||||||
|
// the store's behaviour.)
|
||||||
|
val cutoff = asOf - 1
|
||||||
|
val targets = byId.values.filter { it.flow.value.isExpirationBefore(cutoff) }
|
||||||
|
if (targets.isEmpty()) return false
|
||||||
|
var changed = false
|
||||||
|
for (slot in targets) {
|
||||||
|
if (removeSlot(slot)) changed = true
|
||||||
|
}
|
||||||
|
return changed
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Returns true if the event matches the filter and its arrival
|
* Returns true if processing the event caused membership of the
|
||||||
* caused membership to change (a fresh slot was added). Returns
|
* projection's [items] list to change. Returns false for in-place
|
||||||
* false when the arrival was an in-place supersession update or
|
* supersession updates, NIP-01 tiebreaker rejections, and arrivals
|
||||||
* was rejected by the NIP-01 tiebreaker.
|
* that no filter matches.
|
||||||
*/
|
*/
|
||||||
@Suppress("UNCHECKED_CAST")
|
@Suppress("UNCHECKED_CAST")
|
||||||
private fun handleInsert(event: Event): Boolean {
|
private fun handleInsert(event: Event): Boolean {
|
||||||
val address = addressOf(event)
|
val address = addressOf(event)
|
||||||
|
|
||||||
if (address != null) {
|
if (address != null) {
|
||||||
val existing = byAddress[address]
|
val existing = byAddress[address]
|
||||||
if (existing != null) {
|
if (existing != null) {
|
||||||
if (!supersedes(event, existing.flow.value)) return false
|
if (!supersedes(event, existing.flow.value)) return false
|
||||||
|
|
||||||
// Same address, new winner. Rekey byId and update the
|
// Same address, new winner. Rekey byId from the
|
||||||
|
// previous event id to the new one and update the
|
||||||
// handle's value in place — list reference stays the
|
// handle's value in place — list reference stays the
|
||||||
// same; only the handle's collectors re-render.
|
// same; only the handle's collectors re-render.
|
||||||
val previousId = existing.flow.value.id
|
val previousId = existing.flow.value.id
|
||||||
@@ -258,7 +277,34 @@ class EventStoreProjection<T : Event>(
|
|||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
|
|
||||||
insertNew(event)
|
// Genuinely new slot. Offer it to every matching filter; if
|
||||||
|
// any retains it, the slot becomes live.
|
||||||
|
val slot = Slot(event as T)
|
||||||
|
var retained = false
|
||||||
|
for ((f, set) in perFilter) {
|
||||||
|
if (!f.match(event)) continue
|
||||||
|
set.add(slot)
|
||||||
|
retained = true
|
||||||
|
val cap = f.limit ?: continue
|
||||||
|
while (set.size > cap) {
|
||||||
|
val tail = set.last()
|
||||||
|
set.remove(tail)
|
||||||
|
if (tail === slot) {
|
||||||
|
// We were evicted by our own filter before any
|
||||||
|
// other filter got a chance to retain us — the
|
||||||
|
// remaining loop iterations may still pick us up.
|
||||||
|
retained = false
|
||||||
|
} else if (perFilter.values.none { it.contains(tail) }) {
|
||||||
|
// Tail no longer retained by any filter — drop it.
|
||||||
|
dropSlotIndexes(tail)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (!retained) return false
|
||||||
|
|
||||||
|
byId[event.id] = slot
|
||||||
|
if (address != null) byAddress[address] = slot
|
||||||
|
ordered.add(slot)
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -275,11 +321,11 @@ class EventStoreProjection<T : Event>(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// NIP-09: delete by address, only original author, only events
|
// NIP-09: delete by address, only original author, only events
|
||||||
// with `created_at <= deletion.created_at`.
|
// with `created_at <= deletion.created_at`. `addr` is already
|
||||||
|
// an [Address] — same equals/hashCode as our index keys.
|
||||||
for (addr in deletion.deleteAddresses()) {
|
for (addr in deletion.deleteAddresses()) {
|
||||||
if (addr.pubKeyHex != deletion.pubKey) continue
|
if (addr.pubKeyHex != deletion.pubKey) continue
|
||||||
val key = addressOf(addr.kind, addr.pubKeyHex, addr.dTag) ?: continue
|
val slot = byAddress[addr] ?: continue
|
||||||
val slot = byAddress[key] ?: continue
|
|
||||||
if (slot.flow.value.createdAt <= deletion.createdAt) {
|
if (slot.flow.value.createdAt <= deletion.createdAt) {
|
||||||
if (removeSlot(slot)) changed = true
|
if (removeSlot(slot)) changed = true
|
||||||
}
|
}
|
||||||
@@ -302,33 +348,33 @@ class EventStoreProjection<T : Event>(
|
|||||||
return changed
|
return changed
|
||||||
}
|
}
|
||||||
|
|
||||||
@Suppress("UNCHECKED_CAST")
|
/** Drop a live slot from every index AND from each per-filter set. */
|
||||||
private fun insertNew(event: Event) {
|
|
||||||
val slot = Slot(event as T)
|
|
||||||
byId[event.id] = slot
|
|
||||||
addressOf(event)?.let { byAddress[it] = slot }
|
|
||||||
ordered.add(slot)
|
|
||||||
|
|
||||||
val cap = limit ?: return
|
|
||||||
while (ordered.size > cap) {
|
|
||||||
val tail = ordered.last()
|
|
||||||
removeSlot(tail)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private fun removeSlot(slot: Slot<T>): Boolean {
|
private fun removeSlot(slot: Slot<T>): Boolean {
|
||||||
val removed = byId.remove(slot.flow.value.id) != null
|
val removed = byId.remove(slot.flow.value.id) != null
|
||||||
if (!removed) return false
|
if (!removed) return false
|
||||||
ordered.remove(slot)
|
ordered.remove(slot)
|
||||||
addressOf(slot.flow.value)?.let { address ->
|
addressOf(slot.flow.value)?.let { address ->
|
||||||
// Defensive: only clear the address index if this slot
|
// Defensive: only clear the address index if this slot
|
||||||
// still owns it. Could be stale after an addressable rekey
|
// still owns it.
|
||||||
// raced with another insert.
|
|
||||||
if (byAddress[address] === slot) byAddress.remove(address)
|
if (byAddress[address] === slot) byAddress.remove(address)
|
||||||
}
|
}
|
||||||
|
for (set in perFilter.values) set.remove(slot)
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Drop a slot from byId / byAddress / ordered without touching the
|
||||||
|
* per-filter sets. Used when the per-filter eviction loop already
|
||||||
|
* owns the bookkeeping for those sets.
|
||||||
|
*/
|
||||||
|
private fun dropSlotIndexes(slot: Slot<T>) {
|
||||||
|
byId.remove(slot.flow.value.id)
|
||||||
|
ordered.remove(slot)
|
||||||
|
addressOf(slot.flow.value)?.let { address ->
|
||||||
|
if (byAddress[address] === slot) byAddress.remove(address)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
private fun publish() {
|
private fun publish() {
|
||||||
_items.value = ordered.map { it.flow }
|
_items.value = ordered.map { it.flow }
|
||||||
}
|
}
|
||||||
@@ -343,6 +389,7 @@ class EventStoreProjection<T : Event>(
|
|||||||
ordered.clear()
|
ordered.clear()
|
||||||
byId.clear()
|
byId.clear()
|
||||||
byAddress.clear()
|
byAddress.clear()
|
||||||
|
for (set in perFilter.values) set.clear()
|
||||||
_items.value = emptyList()
|
_items.value = emptyList()
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -351,7 +398,7 @@ class EventStoreProjection<T : Event>(
|
|||||||
* one of these for as long as it survives. The sort key is frozen
|
* one of these for as long as it survives. The sort key is frozen
|
||||||
* at construction time — supersession in-place updates rewrite
|
* at construction time — supersession in-place updates rewrite
|
||||||
* `flow.value` but never the sort key, so the ordering inside
|
* `flow.value` but never the sort key, so the ordering inside
|
||||||
* [ordered] is stable across updates.
|
* [ordered] / [perFilter] is stable across updates.
|
||||||
*/
|
*/
|
||||||
private class Slot<T : Event>(
|
private class Slot<T : Event>(
|
||||||
initial: T,
|
initial: T,
|
||||||
@@ -367,16 +414,10 @@ class EventStoreProjection<T : Event>(
|
|||||||
* supersession. `null` for regular events (which only collide
|
* supersession. `null` for regular events (which only collide
|
||||||
* on event id).
|
* on event id).
|
||||||
*/
|
*/
|
||||||
private fun addressOf(event: Event): Address? = addressOf(event.kind, event.pubKey, (event as? AddressableEvent)?.dTag())
|
private fun addressOf(event: Event): Address? =
|
||||||
|
|
||||||
private fun addressOf(
|
|
||||||
kind: Int,
|
|
||||||
pubKeyHex: HexKey,
|
|
||||||
dTag: String?,
|
|
||||||
): Address? =
|
|
||||||
when {
|
when {
|
||||||
kind.isAddressable() -> Address(kind, pubKeyHex, dTag ?: "")
|
event is AddressableEvent -> event.address()
|
||||||
kind.isReplaceable() -> Address(kind, pubKeyHex, "")
|
event.kind.isReplaceable() -> Address(event.kind, event.pubKey, "")
|
||||||
else -> null
|
else -> null
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -403,14 +444,6 @@ class EventStoreProjection<T : Event>(
|
|||||||
*/
|
*/
|
||||||
private fun ownerPubKey(event: Event): HexKey = (event as? GiftWrapEvent)?.recipientPubKey() ?: event.pubKey
|
private fun ownerPubKey(event: Event): HexKey = (event as? GiftWrapEvent)?.recipientPubKey() ?: event.pubKey
|
||||||
|
|
||||||
private fun isExpiredAt(
|
|
||||||
event: Event,
|
|
||||||
now: Long,
|
|
||||||
): Boolean {
|
|
||||||
val exp = event.expiration() ?: return false
|
|
||||||
return exp <= now
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* created_at DESC, id ASC. The keys are snapshots taken at
|
* created_at DESC, id ASC. The keys are snapshots taken at
|
||||||
* insertion time, so the ordering of a slot never changes
|
* insertion time, so the ordering of a slot never changes
|
||||||
|
|||||||
+4
-5
@@ -18,14 +18,13 @@
|
|||||||
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||||
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||||
*/
|
*/
|
||||||
package com.vitorpamplona.quartz.nip01Core.store.observable
|
package com.vitorpamplona.quartz.nip01Core.store.projection
|
||||||
|
|
||||||
import com.vitorpamplona.quartz.nip01Core.core.Event
|
import com.vitorpamplona.quartz.nip01Core.core.Event
|
||||||
import com.vitorpamplona.quartz.nip01Core.core.isEphemeral
|
import com.vitorpamplona.quartz.nip01Core.core.isEphemeral
|
||||||
import com.vitorpamplona.quartz.nip01Core.relay.filters.Filter
|
import com.vitorpamplona.quartz.nip01Core.relay.filters.Filter
|
||||||
import com.vitorpamplona.quartz.nip01Core.relay.normalizer.NormalizedRelayUrl
|
import com.vitorpamplona.quartz.nip01Core.relay.normalizer.NormalizedRelayUrl
|
||||||
import com.vitorpamplona.quartz.nip01Core.store.IEventStore
|
import com.vitorpamplona.quartz.nip01Core.store.IEventStore
|
||||||
import com.vitorpamplona.quartz.nip01Core.store.projection.EventStoreProjection
|
|
||||||
import com.vitorpamplona.quartz.nip40Expiration.isExpired
|
import com.vitorpamplona.quartz.nip40Expiration.isExpired
|
||||||
import com.vitorpamplona.quartz.utils.TimeUtils
|
import com.vitorpamplona.quartz.utils.TimeUtils
|
||||||
import kotlinx.coroutines.CoroutineScope
|
import kotlinx.coroutines.CoroutineScope
|
||||||
@@ -144,12 +143,12 @@ class ObservableEventStore(
|
|||||||
|
|
||||||
override suspend fun delete(filter: Filter) {
|
override suspend fun delete(filter: Filter) {
|
||||||
inner.delete(filter)
|
inner.delete(filter)
|
||||||
_events.emit(StoreEvent.Delete(DeleteRule.Filtered(listOf(filter))))
|
_events.emit(StoreEvent.DeleteByFilter(listOf(filter)))
|
||||||
}
|
}
|
||||||
|
|
||||||
override suspend fun delete(filters: List<Filter>) {
|
override suspend fun delete(filters: List<Filter>) {
|
||||||
inner.delete(filters)
|
inner.delete(filters)
|
||||||
_events.emit(StoreEvent.Delete(DeleteRule.Filtered(filters)))
|
_events.emit(StoreEvent.DeleteByFilter(filters))
|
||||||
}
|
}
|
||||||
|
|
||||||
override suspend fun deleteExpiredEvents() {
|
override suspend fun deleteExpiredEvents() {
|
||||||
@@ -160,7 +159,7 @@ class ObservableEventStore(
|
|||||||
// own clock when the event is processed.
|
// own clock when the event is processed.
|
||||||
val asOf = TimeUtils.now()
|
val asOf = TimeUtils.now()
|
||||||
inner.deleteExpiredEvents()
|
inner.deleteExpiredEvents()
|
||||||
_events.emit(StoreEvent.Delete(DeleteRule.Expired(asOf)))
|
_events.emit(StoreEvent.DeleteExpired(asOf))
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
+16
-30
@@ -18,7 +18,7 @@
|
|||||||
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||||
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||||
*/
|
*/
|
||||||
package com.vitorpamplona.quartz.nip01Core.store.observable
|
package com.vitorpamplona.quartz.nip01Core.store.projection
|
||||||
|
|
||||||
import com.vitorpamplona.quartz.nip01Core.core.Event
|
import com.vitorpamplona.quartz.nip01Core.core.Event
|
||||||
import com.vitorpamplona.quartz.nip01Core.relay.filters.Filter
|
import com.vitorpamplona.quartz.nip01Core.relay.filters.Filter
|
||||||
@@ -32,40 +32,26 @@ import com.vitorpamplona.quartz.nip01Core.relay.filters.Filter
|
|||||||
* layer (persistable or ephemeral). Carries the event itself so
|
* layer (persistable or ephemeral). Carries the event itself so
|
||||||
* the projection can run its NIP-01 / NIP-09 / NIP-62
|
* the projection can run its NIP-01 / NIP-09 / NIP-62
|
||||||
* interpretation.
|
* interpretation.
|
||||||
* - [Delete] is emitted for every out-of-band removal — manual
|
* - [DeleteByFilter] is emitted for every `delete(filter)` /
|
||||||
* `delete(filter)` / `delete(filters)` calls and the periodic
|
* `delete(filters)` call on the observable. Carries the same
|
||||||
* `deleteExpiredEvents()` sweep. Carries the rule the store used
|
* filters the store used so projections can apply
|
||||||
* so the projection can apply the same selection in memory
|
* [Filter.match] in memory and drop the matching slots without
|
||||||
* without re-querying.
|
* re-querying.
|
||||||
|
* - [DeleteExpired] is emitted for every `deleteExpiredEvents()`
|
||||||
|
* sweep. The optional [DeleteExpired.asOf] cutoff lets the store
|
||||||
|
* pin the timestamp it actually used, so the projection drops
|
||||||
|
* exactly the events the store dropped.
|
||||||
*/
|
*/
|
||||||
sealed interface StoreEvent {
|
sealed interface StoreEvent {
|
||||||
data class Insert(
|
data class Insert(
|
||||||
val event: Event,
|
val event: Event,
|
||||||
) : StoreEvent
|
) : StoreEvent
|
||||||
|
|
||||||
data class Delete(
|
data class DeleteByFilter(
|
||||||
val rule: DeleteRule,
|
val filters: List<Filter>,
|
||||||
|
) : StoreEvent
|
||||||
|
|
||||||
|
data class DeleteExpired(
|
||||||
|
val asOf: Long? = null,
|
||||||
) : StoreEvent
|
) : StoreEvent
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Selection rule for a [StoreEvent.Delete]. [Filtered] mirrors the
|
|
||||||
* `delete(filter)` / `delete(filters)` API and OR-combines the
|
|
||||||
* filters; [Expired] mirrors `deleteExpiredEvents()` and removes
|
|
||||||
* everything whose NIP-40 expiration has lapsed.
|
|
||||||
*/
|
|
||||||
sealed interface DeleteRule {
|
|
||||||
data class Filtered(
|
|
||||||
val filters: List<Filter>,
|
|
||||||
) : DeleteRule
|
|
||||||
|
|
||||||
/**
|
|
||||||
* NIP-40 expiration sweep. Optional [asOf] timestamp (unix
|
|
||||||
* seconds) lets the store pin the cutoff it actually used, so
|
|
||||||
* the projection's in-memory drop matches the store's on-disk
|
|
||||||
* drop exactly. When `null` the projection uses its own clock.
|
|
||||||
*/
|
|
||||||
data class Expired(
|
|
||||||
val asOf: Long? = null,
|
|
||||||
) : DeleteRule
|
|
||||||
}
|
|
||||||
+64
-2
@@ -25,7 +25,7 @@ import com.vitorpamplona.quartz.nip01Core.metadata.MetadataEvent
|
|||||||
import com.vitorpamplona.quartz.nip01Core.relay.filters.Filter
|
import com.vitorpamplona.quartz.nip01Core.relay.filters.Filter
|
||||||
import com.vitorpamplona.quartz.nip01Core.relay.normalizer.normalizeRelayUrl
|
import com.vitorpamplona.quartz.nip01Core.relay.normalizer.normalizeRelayUrl
|
||||||
import com.vitorpamplona.quartz.nip01Core.signers.NostrSignerSync
|
import com.vitorpamplona.quartz.nip01Core.signers.NostrSignerSync
|
||||||
import com.vitorpamplona.quartz.nip01Core.store.observable.ObservableEventStore
|
import com.vitorpamplona.quartz.nip01Core.store.projection.ObservableEventStore
|
||||||
import com.vitorpamplona.quartz.nip01Core.store.sqlite.EventStore
|
import com.vitorpamplona.quartz.nip01Core.store.sqlite.EventStore
|
||||||
import com.vitorpamplona.quartz.nip09Deletions.DeletionEvent
|
import com.vitorpamplona.quartz.nip09Deletions.DeletionEvent
|
||||||
import com.vitorpamplona.quartz.nip10Notes.TextNoteEvent
|
import com.vitorpamplona.quartz.nip10Notes.TextNoteEvent
|
||||||
@@ -442,6 +442,68 @@ class EventStoreProjectionTest {
|
|||||||
projection.close()
|
projection.close()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Per-filter limit: filter A caps at 2, filter B caps at 2, but
|
||||||
|
* the events they match are disjoint, so the projection's union
|
||||||
|
* is 4 (larger than any single filter's cap).
|
||||||
|
*/
|
||||||
|
@Test
|
||||||
|
fun perFilterLimitUnionExceedsSingleLimit() =
|
||||||
|
runBlocking {
|
||||||
|
val authorA = NostrSignerSync()
|
||||||
|
val authorB = NostrSignerSync()
|
||||||
|
val a1 = authorA.sign(TextNoteEvent.build("a1", createdAt = 100))
|
||||||
|
val a2 = authorA.sign(TextNoteEvent.build("a2", createdAt = 200))
|
||||||
|
val b1 = authorB.sign(TextNoteEvent.build("b1", createdAt = 110))
|
||||||
|
val b2 = authorB.sign(TextNoteEvent.build("b2", createdAt = 210))
|
||||||
|
observable.insert(a1)
|
||||||
|
observable.insert(a2)
|
||||||
|
observable.insert(b1)
|
||||||
|
observable.insert(b2)
|
||||||
|
|
||||||
|
val filterA = Filter(kinds = listOf(TextNoteEvent.KIND), authors = listOf(authorA.pubKey), limit = 2)
|
||||||
|
val filterB = Filter(kinds = listOf(TextNoteEvent.KIND), authors = listOf(authorB.pubKey), limit = 2)
|
||||||
|
val projection =
|
||||||
|
observable.observe<TextNoteEvent>(
|
||||||
|
listOf(filterA, filterB),
|
||||||
|
store.relay,
|
||||||
|
scope,
|
||||||
|
)
|
||||||
|
projection.ready.await()
|
||||||
|
assertEquals(4, projection.items.value.size, "per-filter caps don't dedupe union")
|
||||||
|
projection.close()
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Each filter's cap is enforced on its own retained set: filter A
|
||||||
|
* keeps only its 2 newest matches even when more arrive.
|
||||||
|
*/
|
||||||
|
@Test
|
||||||
|
fun perFilterLimitEvictsOldestWithinFilter() =
|
||||||
|
runBlocking {
|
||||||
|
val a1 = signer.sign(TextNoteEvent.build("a1", createdAt = 100))
|
||||||
|
val a2 = signer.sign(TextNoteEvent.build("a2", createdAt = 200))
|
||||||
|
val a3 = signer.sign(TextNoteEvent.build("a3", createdAt = 300))
|
||||||
|
observable.insert(a1)
|
||||||
|
observable.insert(a2)
|
||||||
|
|
||||||
|
val projection =
|
||||||
|
observable.observe<TextNoteEvent>(
|
||||||
|
Filter(kinds = listOf(TextNoteEvent.KIND), limit = 2),
|
||||||
|
store.relay,
|
||||||
|
scope,
|
||||||
|
)
|
||||||
|
projection.ready.await()
|
||||||
|
assertEquals(2, projection.items.value.size)
|
||||||
|
|
||||||
|
observable.insert(a3)
|
||||||
|
val after = projection.awaitItems { it[0].value.id == a3.id }
|
||||||
|
assertEquals(2, after.size)
|
||||||
|
assertEquals(a3.id, after[0].value.id)
|
||||||
|
assertEquals(a2.id, after[1].value.id)
|
||||||
|
projection.close()
|
||||||
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun closeStopsListening() =
|
fun closeStopsListening() =
|
||||||
runBlocking {
|
runBlocking {
|
||||||
@@ -461,7 +523,7 @@ class EventStoreProjectionTest {
|
|||||||
* Ephemeral events (kinds `20000-29999`) are never persisted, so
|
* Ephemeral events (kinds `20000-29999`) are never persisted, so
|
||||||
* the inner SQLite store silently drops them. The
|
* the inner SQLite store silently drops them. The
|
||||||
* `ObservableEventStore` wrapper still routes them onto its
|
* `ObservableEventStore` wrapper still routes them onto its
|
||||||
* [events][com.vitorpamplona.quartz.nip01Core.store.observable.ObservableEventStore.events]
|
* [events][com.vitorpamplona.quartz.nip01Core.store.projection.ObservableEventStore.events]
|
||||||
* flow, so an open projection sees them while it's alive. They
|
* flow, so an open projection sees them while it's alive. They
|
||||||
* vanish from any future seed because the DB never had them.
|
* vanish from any future seed because the DB never had them.
|
||||||
*/
|
*/
|
||||||
|
|||||||
Reference in New Issue
Block a user