nick/amethyst
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Quick event store using SQLite directly.

Browse Source
This commit is contained in:
Vitor Pamplona
2025-07-28 16:49:39 -04:00
parent 88f9beafd3
commit 7ba2810423
27 changed files with 3255 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
quartz/src/main/java/com/vitorpamplona/quartz/nip01Core/store/README.md
+193
View File
@@ -0,0 +1,193 @@
# Event Store Module
This module implements an **Event Store** with nostr-native queries.
The goal was not to make the fastest database, since there could be multiple optimizations made if
consistency can be sacrificed, but a database that will never crash and never go corrupt.
## Responsibilities
- **Storage & Retrieval**:
Stores Nostr events and enables retrieval using Nostr filters
- **Replaceable Events**:
- Old versions are removed when newer versions arrive.
- Old versions are blocked if newer versions exist.
- **Ephemeral Events**
- Ephemeral events never stored.
- **NIP-40 Expirations**
- Manages expiration timestamps and prunes expired events.
- Blocks expired events from being reinserted
- **NIP-09 Deletion Events**
- Deletes by event id
- Deletes by address until the `created_at`
- Blocks deleted events from being re-inserted.
- **NIP-62 Right to Vanish**
- Supports deleting an entire user until the `created_at` for enhanced privacy
- **NIP-50 Full Text Search**:
- Implements content indexing and full text search supporting rich queries over event content.
- **Immutable Tables**
Triggers ensure event immutability.
## Indexing Strategy
The store indexes events using five dedicated tables:
- `event_headers`: stores the canonical event fields.
- `event_tags`: indexes tag values for fast filtering on tag-based queries.
- `event_fts`: for the content of full text search
- `event_expirations`: to control when expired events must be deleted.
- `event_vanish`: to control up to when vanished accounts must be blocked.
SQL triggers ensure the **immutability of stored events**, preventing accidental or intentional
modifications.
## Querying
This module supports optimized query planning, producing efficient SQL for multi-filter evaluation
across fields and tags, including `limit` clauses per filter:
For instance, the following filter:
```kotlin
store.query(
listOf(
Filter(limit = 10),
Filter(authors = listOf(hexkey), kinds = listOf(1, 1111), search = "keywords", limit = 100),
Filter(kinds = listOf(20), search = "cats", limit = 30),
)
)
```
Becomes
```sql
SELECT id, pubkey, created_at, kind, tags, content, sig FROM event_headers
INNER JOIN (
SELECT event_headers.row_id AS row_id
FROM event_headers
ORDER BY created_at DESC,id ASC
LIMIT 10
UNION
SELECT event_headers.row_id AS row_id
FROM event_headers
INNER JOIN event_fts ON event_fts.event_header_row_id = event_headers.row_id
WHERE event_headers.kind IN (1, 1111)
AND event_headers.pubkey = hexkey
AND event_fts MATCH "keywords"
ORDER BY created_at DESC, id ASC
LIMIT 100
UNION
SELECT event_headers.row_id AS row_id
FROM event_headers
INNER JOIN event_fts ON event_fts.event_header_row_id = event_headers.row_id
WHERE event_headers.kind = 20
AND event_fts MATCH "cats"
ORDER BY created_at DESC,id ASC
LIMIT 30
) AS filtered ON event_headers.row_id = filtered.row_id
ORDER BY created_at DESC,id
```
The union operations support complex filter lists while avoiding redundant data fetching and
duplicated outstreams.
## How to Use
The `EventStore` class provides a high-level interface for interacting with the event database.
It is initialized with a `SQLiteDatabase` instance, and it manages the underlying tables and query planning.
### Initialization
To initialize the `EventStore` in your Application class:
```kotlin
val eventStore = EventStore(context, "dbname.db", relayUrlIdentifier)
```
### Querying Events
To query events, use the `query` method with one or more `Filter` objects:
```kotlin
val filters = listOf(
Filter(limit = 10),
Filter(authors = listOf(hexkey))
)
val events = eventStore.query(filters)
```
or to receive events as the cursor sends:
```kotlin
val filters = listOf(
Filter(limit = 10),
Filter(authors = listOf(hexkey))
)
eventStore.query(filters) { event ->
// do something
}
```
`count` and `delete` also accept one or more filters.
### Inserting Events
Insert a single event using the `insert` method:
```kotlin
eventStore.insert(event)
```
### Deleting Events
Events should be deleted by adding a DeletionRequest or a VanishRequest to the db, but to manually
delete an event by ID, use the `delete` method:
```kotlin
eventStore.delete(event.id)
```
### Full-Text Search
The store supports full-text search using the `search` parameter in filters:
```kotlin
val result = eventStore.query(Filter(search = "bitcoin", limit = 20))
```
This will match any event whose content contains "bitcoin", returning the most recent 20 results.
### Periodic cleanup of expired events.
The store exposes a `deleteExpiredEvents` to be used in a periodic clean up procedure. Users
should use a WorkManager or a coroutine to periodically call `store.deleteExpiredEvents()`. We
recommend a 15-minute window to remove recently expired events from the database.
Here's an example of a Worker that should be added to your application class.
```kotlin
class ExpirationWorker(appContext: Context, workerParams: WorkerParameters) : Worker(appContext, workerParams) {
override fun doWork(): Result {
YourApplication.store.deleteExpiredEvents()
return Result.success()
}
}
fun schedulePeriodicWork(context: Context) {
val periodicWorkRequest =
PeriodicWorkRequestBuilder<ExpirationWorker>(15, TimeUnit.MINUTES).build()
WorkManager.getInstance(context).enqueue(periodicWorkRequest)
}
```
quartz/src/main/java/com/vitorpamplona/quartz/nip01Core/store/sqlite/AddressableModule.kt
+72
View File
@@ -0,0 +1,72 @@
/**
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.quartz.nip01Core.store.sqlite
import android.database.sqlite.SQLiteDatabase
class AddressableModule {
fun create(db: SQLiteDatabase) {
db.execSQL(
"""
CREATE UNIQUE INDEX addressable_idx
ON event_headers (kind, pubkey, d_tag)
WHERE kind >= 30000 AND kind < 40000
""".trimIndent(),
)
// Rejects old addressables
db.execSQL(
"""
CREATE TRIGGER reject_older_addressable_event
BEFORE INSERT ON event_headers
FOR EACH ROW
WHEN (NEW.kind >= 30000 AND NEW.kind < 40000)
BEGIN
-- Check for existing newer record
SELECT RAISE(ABORT, 'duplicate: A newer or equally new record already exists')
WHERE EXISTS (
SELECT 1 FROM event_headers
WHERE
event_headers.created_at >= NEW.created_at AND
event_headers.kind = NEW.kind AND
event_headers.pubkey = NEW.pubkey AND
event_headers.d_tag = NEW.d_tag
);
DELETE FROM event_tags
WHERE event_header_row_id in (
SELECT row_id FROM event_headers
WHERE
event_headers.kind = NEW.kind AND
event_headers.pubkey = NEW.pubkey AND
event_headers.d_tag = NEW.d_tag
);
DELETE FROM event_headers
WHERE
event_headers.kind = NEW.kind AND
event_headers.pubkey = NEW.pubkey AND
event_headers.d_tag = NEW.d_tag;
END;
""".trimIndent(),
)
}
}
quartz/src/main/java/com/vitorpamplona/quartz/nip01Core/store/sqlite/DeletionRequestModule.kt
+98
View File
@@ -0,0 +1,98 @@
/**
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.quartz.nip01Core.store.sqlite
import android.database.sqlite.SQLiteDatabase
import com.vitorpamplona.quartz.nip01Core.core.Event
import com.vitorpamplona.quartz.nip09Deletions.DeletionEvent
class DeletionRequestModule {
fun create(db: SQLiteDatabase) {
// rejects deleted events.
db.execSQL(
"""
CREATE TRIGGER reject_deleted_events
BEFORE INSERT ON event_headers
FOR EACH ROW
BEGIN
-- Check for ID-based deletion record
SELECT RAISE(ABORT, 'blocked: a deletion event for this event id exists')
WHERE EXISTS (
SELECT 1 FROM event_headers INNER JOIN event_tags ON event_headers.row_id = event_tags.event_header_row_id
WHERE
event_headers.created_at >= NEW.created_at AND
event_headers.kind = 5 AND
event_headers.pubkey = NEW.pubkey AND
event_tags.tag_name = 'e' AND
event_tags.tag_value = NEW.id
);
-- Check for address-based deletion record
SELECT RAISE(ABORT, 'blocked: a deletion event for this address exists')
WHERE EXISTS (
SELECT 1 FROM event_headers INNER JOIN event_tags ON event_headers.row_id = event_tags.event_header_row_id
WHERE
event_headers.created_at >= NEW.created_at AND
event_headers.kind = 5 AND
event_headers.pubkey = NEW.pubkey AND
event_tags.tag_name = 'a' AND
event_tags.tag_value = NEW.kind || ':' || NEW.pubkey || ':' || NEW.d_tag
);
END;
""".trimIndent(),
)
}
fun insert(
event: Event,
headerId: Long,
db: SQLiteDatabase,
) {
if (event is DeletionEvent) {
val idValues = event.deleteEventIds()
val idParams = idValues.joinToString(",") { "?" }
val addresses = event.deleteAddresses()
val addressParams = addresses.joinToString(",") { "(?, ?)" }
val addressValues = addresses.flatMap { listOf<Any>(it.kind, it.dTag) }
val whereClause =
if (idValues.isNotEmpty() && addresses.isNotEmpty()) {
"(id IN ($idParams) OR (kind, d_tag) IN ($addressParams)) AND pubkey = ?"
} else if (idValues.isNotEmpty()) {
"id IN ($idParams) AND pubkey = ?"
} else if (addresses.isNotEmpty()) {
"(kind, d_tag) IN ($addressParams) AND pubkey = ?"
} else {
return
}
val whereParams = idValues.plus(addressValues).plus(event.pubKey).toTypedArray()
db.execSQL(
"""
DELETE FROM event_headers
WHERE $whereClause;
""".trimIndent(),
whereParams,
)
}
}
}
quartz/src/main/java/com/vitorpamplona/quartz/nip01Core/store/sqlite/EphemeralModule.kt
+40
View File
@@ -0,0 +1,40 @@
/**
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.quartz.nip01Core.store.sqlite
import android.database.sqlite.SQLiteDatabase
class EphemeralModule {
fun create(db: SQLiteDatabase) {
// Rejects all ephemeral events.
db.execSQL(
"""
CREATE TRIGGER reject_ephemeral_events
BEFORE INSERT ON event_headers
FOR EACH ROW
WHEN (NEW.kind >= 20000 AND NEW.kind < 30000)
BEGIN
SELECT RAISE(ABORT, 'blocked: cannot store ephemeral events');
END;
""".trimIndent(),
)
}
}
quartz/src/main/java/com/vitorpamplona/quartz/nip01Core/store/sqlite/EventIndexesModule.kt
+469
View File
@@ -0,0 +1,469 @@
/**
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.quartz.nip01Core.store.sqlite
import android.database.Cursor
import android.database.sqlite.SQLiteDatabase
import com.fasterxml.jackson.module.kotlin.readValue
import com.vitorpamplona.quartz.EventFactory
import com.vitorpamplona.quartz.nip01Core.core.AddressableEvent
import com.vitorpamplona.quartz.nip01Core.core.Event
import com.vitorpamplona.quartz.nip01Core.core.Tag
import com.vitorpamplona.quartz.nip01Core.jackson.JsonMapper
import com.vitorpamplona.quartz.nip01Core.relay.filters.Filter
import com.vitorpamplona.quartz.nip01Core.store.sqlite.sql.where
class EventIndexesModule(
val fts: FullTextSearchModule,
) {
fun create(db: SQLiteDatabase) {
db.execSQL(
"""
CREATE TABLE event_headers (
row_id INTEGER PRIMARY KEY AUTOINCREMENT,
id TEXT NOT NULL,
pubkey TEXT NOT NULL,
created_at INTEGER NOT NULL,
kind INTEGER NOT NULL,
d_tag TEXT,
tags TEXT NOT NULL,
content TEXT NOT NULL,
sig TEXT NOT NULL
)
""".trimIndent(),
)
db.execSQL(
"""
CREATE TABLE event_tags (
event_header_row_id INTEGER,
tag_name TEXT NOT NULL,
tag_value TEXT NOT NULL,
FOREIGN KEY (event_header_row_id) REFERENCES event_headers(row_id) ON DELETE CASCADE
)
""".trimIndent(),
)
db.execSQL("CREATE UNIQUE INDEX event_headers_id ON event_headers (id)")
db.execSQL("CREATE INDEX query_by_kind_pubkey_idx ON event_headers (created_at desc, kind, pubkey, d_tag)")
db.execSQL("CREATE INDEX query_by_id_idx ON event_headers (created_at desc, id)")
db.execSQL("CREATE INDEX query_by_tags_idx ON event_tags (tag_name, tag_value)")
// Prevent updates to maintain immutability
db.execSQL(
"""
CREATE TRIGGER event_headers_prevent_update
BEFORE UPDATE ON event_headers
FOR EACH ROW
BEGIN
SELECT RAISE(ABORT, 'Error: Updates are not allowed.');
END;
""".trimIndent(),
)
db.execSQL(
"""
CREATE TRIGGER event_tags_prevent_update
BEFORE UPDATE ON event_tags
FOR EACH ROW
BEGIN
SELECT RAISE(ABORT, 'Error: Updates are not allowed.');
END;
""".trimIndent(),
)
}
val sqlInsertHeader =
"""
INSERT INTO event_headers
(id, pubkey, created_at, kind, tags, content, sig, d_tag)
VALUES
(?, ?, ?, ?, ?, ?, ?, ?)
""".trimIndent()
val sqlInsertTags =
"""
INSERT OR ROLLBACK INTO event_tags
(event_header_row_id, tag_name, tag_value)
VALUES
(?,?,?)
""".trimIndent()
fun insert(
event: Event,
db: SQLiteDatabase,
): Long {
val stmt = db.compileStatement(sqlInsertHeader)
stmt.bindString(1, event.id)
stmt.bindString(2, event.pubKey)
stmt.bindLong(3, event.createdAt)
stmt.bindLong(4, event.kind.toLong())
stmt.bindString(5, JsonMapper.mapper.writeValueAsString(event.tags))
stmt.bindString(6, event.content)
stmt.bindString(7, event.sig)
if (event is AddressableEvent) {
stmt.bindString(8, event.dTag())
} else {
stmt.bindNull(8)
}
val headerId = stmt.executeInsert()
val tagsToIndex = event.indexableTags()
if (tagsToIndex.isNotEmpty()) {
val sql =
buildString {
append(sqlInsertTags)
repeat(tagsToIndex.size - 1) {
append(",(?,?,?)")
}
}
val stmtTags = db.compileStatement(sql)
var index = 1
tagsToIndex.forEach { tag ->
stmtTags.bindLong(index++, headerId)
stmtTags.bindString(index++, tag[0])
stmtTags.bindString(index++, tag[1])
}
stmtTags.executeInsert()
}
return headerId
}
fun Event.indexableTags(): List<Tag> {
val indexableTagNames = extraIndexableTagNames()
return if (indexableTagNames.isNotEmpty()) {
tags.filter { it.size >= 2 && (it[0].length == 1 || it[0] in indexableTagNames) }
} else {
tags.filter { it.size >= 2 && it[0].length == 1 }
}
}
fun planQuery(filter: Filter): String {
val rowIdSubQuery = prepareRowIDSubQueries(filter) ?: return makeEverythingQuery()
return makeQueryIn(rowIdSubQuery.sql)
}
fun query(
filter: Filter,
db: SQLiteDatabase,
): List<Event> {
val rowIdSubQuery = prepareRowIDSubQueries(filter) ?: return db.runQuery(makeEverythingQuery())
return db.runQuery(makeQueryIn(rowIdSubQuery.sql), rowIdSubQuery.args)
}
fun query(
filter: Filter,
db: SQLiteDatabase,
onEach: (Event) -> Unit,
) {
val rowIdSubQuery = prepareRowIDSubQueries(filter) ?: return db.runQueryEmitting(makeEverythingQuery(), onEach = onEach)
db.runQueryEmitting(makeQueryIn(rowIdSubQuery.sql), rowIdSubQuery.args, onEach)
}
fun planQuery(filters: List<Filter>): String {
val rowIdSubQueries = filters.mapNotNull { prepareRowIDSubQueries(it) }
if (rowIdSubQueries.isEmpty()) return makeEverythingQuery()
val unions = rowIdSubQueries.joinToString(" UNION ") { it.sql }
return makeQueryIn(unions)
}
fun query(
filters: List<Filter>,
db: SQLiteDatabase,
): List<Event> {
val rowIdSubQueries = filters.mapNotNull { prepareRowIDSubQueries(it) }
if (rowIdSubQueries.isEmpty()) return db.runQuery(makeEverythingQuery())
val unions = rowIdSubQueries.joinToString(" UNION ") { it.sql }
val args = rowIdSubQueries.flatMap { it.args }
return db.runQuery(makeQueryIn(unions), args)
}
fun query(
filters: List<Filter>,
db: SQLiteDatabase,
onEach: (Event) -> Unit,
) {
val rowIdSubQueries = filters.mapNotNull { prepareRowIDSubQueries(it) }
if (rowIdSubQueries.isEmpty()) return db.runQueryEmitting(makeEverythingQuery(), onEach = onEach)
val unions = rowIdSubQueries.joinToString(" UNION ") { it.sql }
val args = rowIdSubQueries.flatMap { it.args }
db.runQueryEmitting(makeQueryIn(unions), args, onEach)
}
private fun makeEverythingQuery() = "SELECT id, pubkey, created_at, kind, tags, content, sig FROM event_headers ORDER BY created_at DESC, id"
private fun makeQueryIn(rowIdQuery: String) =
"""
SELECT id, pubkey, created_at, kind, tags, content, sig FROM event_headers
INNER JOIN ($rowIdQuery) AS filtered
ON event_headers.row_id = filtered.row_id
ORDER BY created_at DESC, id
""".trimIndent()
private fun SQLiteDatabase.runQuery(
sql: String,
args: List<String> = emptyList(),
): List<Event> =
rawQuery(sql, args.toTypedArray()).use { cursor ->
parseResults(cursor)
}
private fun parseResults(cursor: Cursor): List<Event> {
val events = ArrayList<Event>()
while (cursor.moveToNext()) {
events.add(
EventFactory.create(
cursor.getString(0).intern(),
cursor.getString(1).intern(),
cursor.getLong(2),
cursor.getInt(3),
JsonMapper.mapper.readValue<Array<Array<String>>>(cursor.getString(4)),
cursor.getString(5).intern(),
cursor.getString(6).intern(),
),
)
}
return events
}
private fun SQLiteDatabase.runQueryEmitting(
sql: String,
args: List<String> = emptyList(),
onEach: (Event) -> Unit,
) = rawQuery(sql, args.toTypedArray()).use { cursor ->
emitResults(cursor, onEach)
}
private fun emitResults(
cursor: Cursor,
onEach: (Event) -> Unit,
) {
while (cursor.moveToNext()) {
onEach(
EventFactory.create(
cursor.getString(0).intern(),
cursor.getString(1).intern(),
cursor.getLong(2),
cursor.getInt(3),
JsonMapper.mapper.readValue<Array<Array<String>>>(cursor.getString(4)),
cursor.getString(5).intern(),
cursor.getString(6).intern(),
),
)
}
}
// --------------
// Counts
// -------------
fun count(
filter: Filter,
db: SQLiteDatabase,
): Int {
val rowIdSubQuery = prepareRowIDSubQueries(filter) ?: return db.countEverything()
return db.countIn(rowIdSubQuery.sql, rowIdSubQuery.args)
}
fun count(
filters: List<Filter>,
db: SQLiteDatabase,
): Int {
val rowIdSubQueries = filters.mapNotNull { prepareRowIDSubQueries(it) }
if (rowIdSubQueries.isEmpty()) return db.countEverything()
val unions = rowIdSubQueries.joinToString(" UNION ") { it.sql }
val args = rowIdSubQueries.flatMap { it.args }
return db.countIn(unions, args)
}
private fun SQLiteDatabase.countEverything() = runCount("SELECT count(*) as count FROM event_headers")
private fun SQLiteDatabase.countIn(
rowIdQuery: String,
args: List<String>,
) = runCount("SELECT COUNT(*) as count FROM ($rowIdQuery)", args)
private fun SQLiteDatabase.runCount(
sql: String,
args: List<String> = emptyList(),
): Int =
rawQuery(sql, args.toTypedArray()).use { cursor ->
cursor.moveToNext()
cursor.getInt(0)
}
// --------------
// Deletes
// -------------
fun delete(
filter: Filter,
db: SQLiteDatabase,
): Int? {
val rowIdQuery = prepareRowIDSubQueries(filter) ?: return null
return db.runDelete(rowIdQuery.sql, rowIdQuery.args)
}
fun delete(
filters: List<Filter>,
db: SQLiteDatabase,
): Int? {
val rowIdSubqueries = filters.mapNotNull { prepareRowIDSubQueries(it) }
if (rowIdSubqueries.isEmpty()) return null
val unions = rowIdSubqueries.joinToString(" UNION ") { it.sql }
val args = rowIdSubqueries.flatMap { it.args }
return db.runDelete(unions, args)
}
private fun SQLiteDatabase.runDelete(
sql: String,
args: List<String> = emptyList(),
): Int = delete("event_headers", "row_id IN ($sql)", args.toTypedArray())
// ----------------------------
// Inner row id selections
// ----------------------------
fun prepareRowIDSubQueries(filter: Filter): RowIdSubQuery? {
if (!filter.isFilledFilter()) return null
val hasHeaders =
with(filter) {
(ids != null && ids.isNotEmpty()) ||
(authors != null && authors.isNotEmpty()) ||
(kinds != null && kinds.isNotEmpty()) ||
(since != null) ||
(until != null) ||
(tags != null && tags.containsKey("d"))
}
val hasSearch = (filter.search != null && filter.search.isNotBlank())
val projection =
buildString {
val anchorColumn: String
val joins = mutableListOf<String>()
if (hasHeaders) {
append("SELECT event_headers.row_id as row_id FROM event_headers")
anchorColumn = "event_headers.row_id"
if (hasSearch) {
joins.add("INNER JOIN ${fts.tableName} ON ${fts.tableName}.${fts.eventHeaderRowIdName} = $anchorColumn")
}
filter.tags?.forEach { (tagName, _) ->
if (tagName != "d") {
joins.add("INNER JOIN event_tags as tag$tagName ON tag$tagName.event_header_row_id = $anchorColumn")
}
}
} else if (hasSearch) {
append("SELECT ${fts.tableName}.${fts.eventHeaderRowIdName} as row_id FROM ${fts.tableName}")
anchorColumn = "${fts.tableName}.${fts.eventHeaderRowIdName}"
filter.tags?.forEach { (tagName, _) ->
if (tagName != "d") {
joins.add("INNER JOIN event_tags as tag$tagName ON tag$tagName.event_header_row_id = $anchorColumn")
}
}
} else {
// has only tags
filter.tags?.forEach { (tagName, _) ->
if (tagName != "d") {
if (isEmpty()) {
append("SELECT tag$tagName.event_header_row_id as row_id FROM event_tags as tag$tagName")
} else {
joins.add("INNER JOIN event_tags as tag$tagName ON tag$tagName.event_header_row_id = tag${tagName.takeLast(1)}.event_header_row_id")
}
}
}
if (isEmpty()) {
// only limit is present
append("SELECT event_headers.row_id as row_id FROM event_headers")
}
}
if (joins.isNotEmpty()) {
append(" ${joins.joinToString(" ")}")
}
}
val clause =
where {
filter.ids?.let { equalsOrIn("event_headers.id", it) }
filter.kinds?.let { equalsOrIn("event_headers.kind", it) }
filter.authors?.let { equalsOrIn("event_headers.pubkey", it) }
filter.since?.let { greaterThanOrEquals("event_headers.created_at", it) }
filter.until?.let { lessThanOrEquals("event_headers.created_at", it) }
filter.tags?.forEach { (tagName, tagValues) ->
if (tagName == "d") {
equalsOrIn("event_headers.d_tag", tagValues)
} else {
equals("tag$tagName.tag_name", tagName)
equalsOrIn("tag$tagName.tag_value", tagValues)
}
}
filter.search?.let { match(fts.tableName, it) }
}
val whereClause =
if (filter.limit != null) {
"${clause.conditions} ORDER BY created_at DESC, id ASC LIMIT ${filter.limit}"
} else {
clause.conditions
}
return RowIdSubQuery("$projection WHERE $whereClause", clause.args)
}
fun deleteAll(db: SQLiteDatabase) {
db.execSQL("DELETE FROM event_tags")
db.execSQL("DELETE FROM event_headers")
}
class RowIdSubQuery(
val sql: String,
val args: List<String>,
)
}
quartz/src/main/java/com/vitorpamplona/quartz/nip01Core/store/sqlite/EventStore.kt
+61
View File
@@ -0,0 +1,61 @@
/**
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.quartz.nip01Core.store.sqlite
import android.content.Context
import com.vitorpamplona.quartz.nip01Core.core.Event
import com.vitorpamplona.quartz.nip01Core.relay.filters.Filter
class EventStore(
context: Context,
dbName: String? = "events.db",
val relayUrl: String? = "wss://quartz.local",
) {
val store = SQLiteEventStore(context, dbName, relayUrl)
fun insert(event: Event) = store.insertEvent(event)
fun query(filter: Filter) = store.query(filter)
fun query(filters: List<Filter>) = store.query(filters)
fun query(
filter: Filter,
onEach: (Event) -> Unit,
) = store.query(filter, onEach)
fun query(
filters: List<Filter>,
onEach: (Event) -> Unit,
) = store.query(filters, onEach)
fun count(filter: Filter) = store.count(filter)
fun count(filters: List<Filter>) = store.count(filters)
fun delete(filter: Filter) = store.delete(filter)
fun delete(filters: List<Filter>) = store.delete(filters)
fun deleteExpiredEvents() = store.deleteExpiredEvents()
fun close() = store.close()
}
quartz/src/main/java/com/vitorpamplona/quartz/nip01Core/store/sqlite/ExpirationModule.kt
+92
View File
@@ -0,0 +1,92 @@
/**
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.quartz.nip01Core.store.sqlite
import android.database.sqlite.SQLiteDatabase
import com.vitorpamplona.quartz.nip01Core.core.Event
import com.vitorpamplona.quartz.nip40Expiration.expiration
class ExpirationModule {
fun create(db: SQLiteDatabase) {
db.execSQL(
"""
CREATE TABLE event_expirations (
event_header_row_id INTEGER,
expiration INTEGER NOT NULL,
FOREIGN KEY (event_header_row_id) REFERENCES event_headers(row_id) ON DELETE CASCADE
)
""".trimIndent(),
)
// Rejects old addressables
db.execSQL(
"""
CREATE TRIGGER reject_expired_events
BEFORE INSERT ON event_expirations
FOR EACH ROW
BEGIN
-- Check for existing newer record
SELECT RAISE(ABORT, 'blocked: this event is expired')
WHERE NEW.expiration <= unixepoch();
END;
""".trimIndent(),
)
db.execSQL("CREATE UNIQUE INDEX events_exp_id ON event_expirations (event_header_row_id)")
}
val insertExpiration =
"""
INSERT OR ROLLBACK INTO event_expirations (event_header_row_id, expiration)
VALUES (?, ?)
""".trimIndent()
fun insert(
event: Event,
headerId: Long,
db: SQLiteDatabase,
) {
val exp = event.expiration()
if (exp != null && exp > 0) {
val stmt = StatementCache.get(insertExpiration, db)
stmt.bindLong(1, headerId)
stmt.bindLong(2, exp)
stmt.executeInsert()
}
}
val deleteExpiredEvents =
"""
DELETE FROM event_headers
WHERE row_id IN (
SELECT event_expirations.event_header_row_id FROM event_expirations
WHERE event_expirations.expiration < unixepoch()
);
""".trimIndent()
fun deleteExpiredEvents(db: SQLiteDatabase) {
StatementCache.get(deleteExpiredEvents, db).execute()
}
fun deleteAll(db: SQLiteDatabase) {
db.execSQL("DELETE FROM event_expirations")
}
}
quartz/src/main/java/com/vitorpamplona/quartz/nip01Core/store/sqlite/FullTextSearchModule.kt
+92
View File
@@ -0,0 +1,92 @@
/**
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.quartz.nip01Core.store.sqlite
import android.database.sqlite.SQLiteDatabase
import android.database.sqlite.SQLiteException
import com.vitorpamplona.quartz.nip01Core.core.Event
import com.vitorpamplona.quartz.nip50Search.SearchableEvent
class FullTextSearchModule {
val tableName = "event_fts"
val eventHeaderRowIdName = "event_header_row_id"
val contentName = "content"
fun create(db: SQLiteDatabase) {
val ftsVersion = FullTextSearchModule().versionFinder(db)
db.execSQL(
"""
CREATE VIRTUAL TABLE $tableName
USING fts$ftsVersion($eventHeaderRowIdName, $contentName)
""",
)
// Foreign key cleanup for full text search
db.execSQL(
"""
CREATE TRIGGER fts_foreign_key
AFTER DELETE ON event_headers
FOR EACH ROW
BEGIN
DELETE FROM $tableName
WHERE old.row_id = $tableName.$eventHeaderRowIdName;
END;
""",
)
}
val insertFTS =
"""
INSERT OR ROLLBACK INTO $tableName ($eventHeaderRowIdName, $contentName)
VALUES (?, ?)
""".trimIndent()
fun insert(
event: Event,
headerId: Long,
db: SQLiteDatabase,
) {
if (event is SearchableEvent) {
val stmt = StatementCache.get(insertFTS, db)
stmt.bindLong(1, headerId)
stmt.bindString(2, event.indexableContent())
stmt.executeInsert()
}
}
fun versionFinder(db: SQLiteDatabase): Int =
try {
try {
db.execSQL("CREATE VIRTUAL TABLE dummy_fts5 USING fts5(dummy)")
5
} catch (e: SQLiteException) {
db.execSQL("CREATE VIRTUAL TABLE dummy_fts4 USING fts4(dummy)")
4
}
} catch (e: SQLiteException) {
db.execSQL("CREATE VIRTUAL TABLE dummy_fts3 USING fts3(dummy)")
3
}
fun deleteAll(db: SQLiteDatabase) {
db.execSQL("DELETE FROM event_fts")
}
}
quartz/src/main/java/com/vitorpamplona/quartz/nip01Core/store/sqlite/ReplaceableModule.kt
+70
View File
@@ -0,0 +1,70 @@
/**
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.quartz.nip01Core.store.sqlite
import android.database.sqlite.SQLiteDatabase
class ReplaceableModule {
fun create(db: SQLiteDatabase) {
db.execSQL(
"""
CREATE UNIQUE INDEX replaceable_idx
ON event_headers (kind, pubkey)
WHERE (kind >= 10000 AND kind < 20000) OR (kind IN (0, 3))
""".trimIndent(),
)
// Rejects old replaceables
db.execSQL(
"""
CREATE TRIGGER reject_older_replaceable_event
BEFORE INSERT ON event_headers
FOR EACH ROW
WHEN (NEW.kind >= 10000 AND NEW.kind < 20000) OR (NEW.kind IN (0, 3))
BEGIN
-- Check for existing newer record
SELECT RAISE(ABORT, 'duplicate: A newer or equally new record already exists')
WHERE EXISTS (
SELECT 1 FROM event_headers
WHERE
event_headers.created_at >= NEW.created_at AND
event_headers.kind = NEW.kind AND
event_headers.pubkey = NEW.pubkey
);
DELETE FROM event_tags
WHERE event_header_row_id in (
SELECT row_id FROM event_headers
WHERE
event_headers.kind = NEW.kind AND
event_headers.pubkey = NEW.pubkey
);
-- Delete older records if this is the newest
DELETE FROM event_headers
WHERE
event_headers.kind = NEW.kind AND
event_headers.pubkey = NEW.pubkey;
END;
""".trimIndent(),
)
}
}
quartz/src/main/java/com/vitorpamplona/quartz/nip01Core/store/sqlite/RightToVanishModule.kt
+111
View File
@@ -0,0 +1,111 @@
/**
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.quartz.nip01Core.store.sqlite
import android.database.sqlite.SQLiteDatabase
import com.vitorpamplona.quartz.nip01Core.core.Event
import com.vitorpamplona.quartz.nip62RequestToVanish.RequestToVanishEvent
class RightToVanishModule {
fun create(db: SQLiteDatabase) {
db.execSQL(
"""
CREATE TABLE event_vanish (
event_header_row_id INTEGER,
pubkey TEXT NOT NULL,
created_at INTEGER NOT NULL,
FOREIGN KEY (event_header_row_id) REFERENCES event_headers(row_id) ON DELETE CASCADE
)
""".trimIndent(),
)
db.execSQL("CREATE UNIQUE INDEX event_vanish_key ON event_vanish (pubkey)")
db.execSQL(
"""
CREATE TRIGGER delete_older_event_vanish
BEFORE INSERT ON event_vanish
FOR EACH ROW
BEGIN
-- Delete older records if this is the newest
DELETE FROM event_vanish
WHERE
event_vanish.created_at < NEW.created_at AND
event_vanish.pubkey = NEW.pubkey;
END;
""".trimIndent(),
)
db.execSQL(
"""
CREATE TRIGGER delete_events_on_event_vanish
AFTER INSERT ON event_vanish
FOR EACH ROW
BEGIN
DELETE FROM event_headers WHERE created_at < NEW.created_at AND pubkey = NEW.pubkey;
END;
""".trimIndent(),
)
// reject new events inside a right to vanish request
db.execSQL(
"""
CREATE TRIGGER reject_events_on_event_vanish
BEFORE INSERT ON event_headers
FOR EACH ROW
BEGIN
SELECT RAISE(ABORT, 'blocked: a request to vanish event exists')
WHERE EXISTS (
SELECT 1 FROM event_vanish
WHERE
event_vanish.created_at >= NEW.created_at AND
event_vanish.pubkey = NEW.pubkey
);
END;
""".trimIndent(),
)
}
val insertRTV =
"""
INSERT OR ROLLBACK INTO event_vanish (event_header_row_id, pubkey, created_at)
VALUES (?, ?, ?)
""".trimIndent()
fun insert(
event: Event,
relayUrl: String?,
headerId: Long,
db: SQLiteDatabase,
) {
if (event is RequestToVanishEvent && event.shouldVanishFrom(relayUrl)) {
val stmt = StatementCache.get(insertRTV, db)
stmt.bindLong(1, headerId)
stmt.bindString(2, event.pubKey)
stmt.bindLong(3, event.createdAt)
stmt.executeInsert()
}
}
fun deleteAll(db: SQLiteDatabase) {
db.execSQL("DELETE FROM event_vanish")
}
}
quartz/src/main/java/com/vitorpamplona/quartz/nip01Core/store/sqlite/SQLiteEventStore.kt
+134
View File
@@ -0,0 +1,134 @@
/**
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.quartz.nip01Core.store.sqlite
import android.content.Context
import android.database.sqlite.SQLiteConstraintException
import android.database.sqlite.SQLiteDatabase
import android.database.sqlite.SQLiteOpenHelper
import androidx.core.database.sqlite.transaction
import com.vitorpamplona.quartz.nip01Core.core.Event
import com.vitorpamplona.quartz.nip01Core.core.HexKey
import com.vitorpamplona.quartz.nip01Core.core.isEphemeral
import com.vitorpamplona.quartz.nip01Core.relay.filters.Filter
import com.vitorpamplona.quartz.nip40Expiration.isExpired
class SQLiteEventStore(
context: Context,
dbName: String? = "events.db",
val relayUrl: String? = null,
) : SQLiteOpenHelper(context, dbName, null, DATABASE_VERSION) {
companion object {
const val DATABASE_VERSION = 1
}
val fullTextSearchModule = FullTextSearchModule()
val eventIndexModule = EventIndexesModule(fullTextSearchModule)
val replaceableModule = ReplaceableModule()
val addressableModule = AddressableModule()
val ephemeralModule = EphemeralModule()
val deletionModule = DeletionRequestModule()
val expirationModule = ExpirationModule()
val rightToVanishModule = RightToVanishModule()
override fun onConfigure(db: SQLiteDatabase) {
super.onConfigure(db)
// makes sure the FKs are sane
db.setForeignKeyConstraintsEnabled(true)
// SQLite implements mutations by appending them to a log, which it occasionally
// compacts into the database. This is called Write-Ahead Logging (WAL)
db.enableWriteAheadLogging()
// The DB can be corrupted if the OS is shutdown before sync, which generally
// doesn't happen on Android
db.execSQL("PRAGMA synchronous = OFF")
}
override fun onCreate(db: SQLiteDatabase) {
eventIndexModule.create(db)
replaceableModule.create(db)
addressableModule.create(db)
ephemeralModule.create(db)
deletionModule.create(db)
expirationModule.create(db)
rightToVanishModule.create(db)
fullTextSearchModule.create(db)
}
override fun onUpgrade(
db: SQLiteDatabase,
oldVersion: Int,
newVersion: Int,
) {}
fun clearDB() {
val db = writableDatabase
fullTextSearchModule.deleteAll(db)
rightToVanishModule.deleteAll(db)
expirationModule.deleteAll(db)
eventIndexModule.deleteAll(db)
}
fun insertEvent(event: Event): Boolean {
if (event.isExpired()) throw SQLiteConstraintException("blocked: Cannot insert an expired event")
if (event.kind.isEphemeral()) return false
val db = writableDatabase
db.transaction {
val headerId = eventIndexModule.insert(event, db)
deletionModule.insert(event, headerId, db)
expirationModule.insert(event, headerId, db)
fullTextSearchModule.insert(event, headerId, db)
rightToVanishModule.insert(event, relayUrl, headerId, db)
}
return true
}
fun query(filter: Filter): List<Event> = eventIndexModule.query(filter, readableDatabase)
fun query(filters: List<Filter>): List<Event> = eventIndexModule.query(filters, readableDatabase)
fun query(
filter: Filter,
onEach: (Event) -> Unit,
) = eventIndexModule.query(filter, readableDatabase, onEach)
fun query(
filters: List<Filter>,
onEach: (Event) -> Unit,
) = eventIndexModule.query(filters, readableDatabase, onEach)
fun count(filter: Filter): Int = eventIndexModule.count(filter, readableDatabase)
fun count(filters: List<Filter>): Int = eventIndexModule.count(filters, readableDatabase)
fun delete(filter: Filter): Int? = eventIndexModule.delete(filter, writableDatabase)
fun delete(filters: List<Filter>): Int? = eventIndexModule.delete(filters, writableDatabase)
fun delete(id: HexKey): Int = writableDatabase.delete("event_headers", "id = ?", arrayOf(id))
fun deleteExpiredEvents() = expirationModule.deleteExpiredEvents(writableDatabase)
}
quartz/src/main/java/com/vitorpamplona/quartz/nip01Core/store/sqlite/StatementCache.kt
+50
View File
@@ -0,0 +1,50 @@
/**
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.quartz.nip01Core.store.sqlite
import android.database.sqlite.SQLiteDatabase
import android.database.sqlite.SQLiteStatement
import android.util.LruCache
object StatementCache {
data class StatementKey(
val sql: String,
val dbHashcode: Int,
)
val cachedStatements = LruCache<StatementKey, SQLiteStatement>(10)
fun get(
sql: String,
db: SQLiteDatabase,
): SQLiteStatement {
val key = StatementKey(sql, db.hashCode())
val cached = cachedStatements.get(key)
return if (cached != null) {
cached.clearBindings()
cached
} else {
val stat = db.compileStatement(sql)
cachedStatements.put(key, stat)
stat
}
}
}
quartz/src/main/java/com/vitorpamplona/quartz/nip01Core/store/sqlite/sql/Condition.kt
+84
View File
@@ -0,0 +1,84 @@
/**
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.quartz.nip01Core.store.sqlite.sql
sealed class Condition {
data class Equals(
val column: String,
val value: Any?,
) : Condition()
data class NotEquals(
val column: String,
val value: Any?,
) : Condition()
data class GreaterThan(
val column: String,
val value: Any,
) : Condition()
data class GreaterThanOrEquals(
val column: String,
val value: Any,
) : Condition()
data class LessThan(
val column: String,
val value: Any,
) : Condition()
data class LessThanOrEquals(
val column: String,
val value: Any,
) : Condition()
data class Like(
val column: String,
val value: String,
) : Condition()
data class Match(
val table: String,
val value: String,
) : Condition()
data class IsNull(
val column: String,
) : Condition()
data class IsNotNull(
val column: String,
) : Condition()
data class In(
val column: String,
val values: List<Any>,
) : Condition()
data class And(
val conditions: List<Condition>,
) : Condition()
data class Or(
val conditions: List<Condition>,
) : Condition()
}
quartz/src/main/java/com/vitorpamplona/quartz/nip01Core/store/sqlite/sql/SqlSelectionBuilder.kt
+106
View File
@@ -0,0 +1,106 @@
/**
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.quartz.nip01Core.store.sqlite.sql
class SqlSelectionBuilder(
private val condition: Condition,
) {
private val selectionArgs = mutableListOf<String>()
fun build(): WhereClause {
selectionArgs.clear() // Clear previous args for a fresh build
val conditions = buildCondition(condition)
return WhereClause(conditions, selectionArgs)
}
/**
* Recursively builds the SQL string for a given condition.
* @param cond The [Condition] to build the SQL string for.
* @return The SQL string representation of the condition.
*/
private fun buildCondition(cond: Condition): String =
when (cond) {
is Condition.Equals -> {
selectionArgs.add(cond.value.toString())
"${cond.column} = ?"
}
is Condition.NotEquals -> {
selectionArgs.add(cond.value.toString())
"${cond.column} != ?"
}
is Condition.GreaterThan -> {
selectionArgs.add(cond.value.toString())
"${cond.column} > ?"
}
is Condition.GreaterThanOrEquals -> {
selectionArgs.add(cond.value.toString())
"${cond.column} >= ?"
}
is Condition.LessThan -> {
selectionArgs.add(cond.value.toString())
"${cond.column} < ?"
}
is Condition.LessThanOrEquals -> {
selectionArgs.add(cond.value.toString())
"${cond.column} <= ?"
}
is Condition.Like -> {
selectionArgs.add(cond.value)
"${cond.column} LIKE ?"
}
is Condition.Match -> {
selectionArgs.add(cond.value)
"${cond.table} MATCH ?"
}
is Condition.IsNull -> {
"${cond.column} IS NULL"
}
is Condition.IsNotNull -> {
"${cond.column} IS NOT NULL"
}
is Condition.In -> {
if (cond.values.isEmpty()) {
// Handle empty IN clause gracefully, perhaps by making it always false
// or throwing an error, depending on desired behavior.
// For now, let's make it an always false condition to avoid SQL errors.
"1 = 0" // Always false
} else {
val placeholders = cond.values.joinToString(", ") { "?" }
cond.values.forEach { selectionArgs.add(it.toString()) }
"${cond.column} IN ($placeholders)"
}
}
is Condition.And -> {
if (cond.conditions.isEmpty()) {
"1 = 1" // Always true for an empty AND
} else {
cond.conditions.joinToString(" AND ") { "(${buildCondition(it)})" }
}
}
is Condition.Or -> {
if (cond.conditions.isEmpty()) {
"1 = 0" // Always false for an empty OR
} else {
cond.conditions.joinToString(" OR ") { "(${buildCondition(it)})" }
}
}
}
}
quartz/src/main/java/com/vitorpamplona/quartz/nip01Core/store/sqlite/sql/WhereClauseBuilder.kt
+120
View File
@@ -0,0 +1,120 @@
/**
* Copyright (c) 2025 Vitor Pamplona
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of
* this software and associated documentation files (the "Software"), to deal in
* the Software without restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
* Software, and to permit persons to whom the Software is furnished to do so,
* subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
* FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
* COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
* AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
package com.vitorpamplona.quartz.nip01Core.store.sqlite.sql
class WhereClauseBuilder {
private val conditions = mutableListOf<Condition>()
fun equals(
column: String,
value: Any?,
) = apply { conditions.add(Condition.Equals(column, value)) }
fun notEquals(
column: String,
value: Any?,
) = apply { conditions.add(Condition.NotEquals(column, value)) }
fun greaterThan(
column: String,
value: Any,
) = apply { conditions.add(Condition.GreaterThan(column, value)) }
fun greaterThanOrEquals(
column: String,
value: Any,
) = apply { conditions.add(Condition.GreaterThanOrEquals(column, value)) }
fun lessThan(
column: String,
value: Any,
) = apply { conditions.add(Condition.LessThan(column, value)) }
fun lessThanOrEquals(
column: String,
value: Any,
) = apply { conditions.add(Condition.LessThanOrEquals(column, value)) }
fun like(
column: String,
pattern: String,
) = apply { conditions.add(Condition.Like(column, pattern)) }
fun match(
table: String,
pattern: String,
) = apply { conditions.add(Condition.Match(table, pattern)) }
fun isNull(column: String) = apply { conditions.add(Condition.IsNull(column)) }
fun isNotNull(column: String) = apply { conditions.add(Condition.IsNotNull(column)) }
fun isIn(
column: String,
values: List<Any>,
) = apply { conditions.add(Condition.In(column, values)) }
fun equalsOrIn(
column: String,
values: List<Any>,
) = apply {
if (values.size == 1) {
equals(column, values.first())
} else {
isIn(column, values)
}
}
fun and(block: WhereClauseBuilder.() -> Unit) =
apply {
val builder = WhereClauseBuilder().apply(block)
val builtCondition = builder.build()
if (builtCondition != null) {
conditions.add(builtCondition)
}
}
fun or(block: WhereClauseBuilder.() -> Unit) =
apply {
val builder = WhereClauseBuilder().apply(block)
val builtCondition = builder.build()
if (builtCondition != null) {
conditions.add(builtCondition)
}
}
fun build(): Condition? =
when (conditions.size) {
0 -> null
1 -> conditions.first()
else -> Condition.And(conditions.toList())
}
}
fun where(block: WhereClauseBuilder.() -> Unit): WhereClause {
val condition = WhereClauseBuilder().apply(block).build() ?: Condition.And(emptyList())
return SqlSelectionBuilder(condition).build()
}
class WhereClause(
val conditions: String,
val args: List<String>,
)