eventium-sqlite 0.1.0 → 0.2.1
raw patch · 7 files changed
+227/−214 lines, 7 filesdep +eventium-testkitdep −eventium-test-helpersdep ~eventium-coredep ~eventium-sql-commondep ~persistent
Dependencies added: eventium-testkit
Dependencies removed: eventium-test-helpers
Dependency ranges changed: eventium-core, eventium-sql-common, persistent
Files
- CHANGELOG.md +17/−0
- README.md +45/−189
- eventium-sqlite.cabal +33/−13
- src/Eventium/ProjectionCache/Sqlite.hs +52/−0
- src/Eventium/Store/Sqlite.hs +16/−7
- tests/Eventium/ProjectionCache/SqliteSpec.hs +59/−0
- tests/Eventium/Store/SqliteSpec.hs +5/−5
CHANGELOG.md view
@@ -0,0 +1,17 @@+# eventium-sqlite Changelog++## 0.2.1 (Unreleased)++- `sqliteCheckpointStore` -- SQLite-backed `CheckpointStore` for `SequenceNumber` tracking.+- Re-exports `CheckpointName` from `eventium-sql-common`.++## 0.2.0 (Unreleased)++- Updated for eventium-core 0.2.0 API changes.+- Single `events` table with metadata columns (event_type, correlation_id, causation_id, created_at).+- `initializeSqliteEventStore` creates UUID index on the events table.+- Enabled `NoFieldSelectors`, `DuplicateRecordFields`, `OverloadedRecordDot` default extensions.++## 0.1.0++Initial release.
README.md view
@@ -1,222 +1,78 @@ # Eventium SQLite -SQLite-based event store implementation for embedded and single-process applications.+SQLite event store backend for Eventium. ## Overview -`eventium-sqlite` provides a lightweight, file-based event store implementation using SQLite. It's perfect for single-process applications, embedded systems, mobile apps, and scenarios where you want persistent storage without the complexity of a database server.--## Features--- ✅ **Zero Configuration** - No database server required-- ✅ **File-Based Storage** - Single database file for easy backup-- ✅ **ACID Transactions** - Full consistency guarantees from SQLite-- ✅ **Optimistic Concurrency** - Version-based conflict detection-- ✅ **Type-Safe Access** - Uses Persistent library-- ✅ **Cross-Platform** - Works on Linux, macOS, Windows-- ✅ **Embedded-Friendly** - Low resource footprint-- ✅ **Easy Deployment** - No separate database process--## When to Use SQLite--### ✅ Good Fit-- **Desktop Applications** - Local data storage-- **CLI Tools** - Persistent command-line applications-- **Mobile Apps** - Embedded event storage-- **Development/Testing** - Persistent data without server setup-- **Single-Process Systems** - No concurrent process access needed-- **Edge Computing** - Resource-constrained environments--### ⚠️ Consider Alternatives-- **Multi-Process Systems** - Use `eventium-postgresql` instead-- **High Write Concurrency** - PostgreSQL handles concurrent writes better-- **Distributed Systems** - Need a client-server database-- **Very Large Datasets** - PostgreSQL scales better for TB+ data--## Installation--Add to your `package.yaml`:--```yaml-dependencies:- - eventium-core- - eventium-sql-common- - eventium-sqlite- - persistent-sqlite # SQLite driver-```--## Usage--```haskell-import Eventium.Store.Sqlite-import Database.Persist.Sqlite--main :: IO ()-main = do- -- Use file-based storage- withSqlitePool "events.db" 1 $ \pool -> do- -- Initialize schema- flip runSqlPool pool $ do- runMigration migrateAll- - -- Create event store- let store = makeSqliteEventStore pool- - -- Use with command handlers- result <- applyCommandHandler - (eventStoreWriter store)- (eventStoreReader store)- commandHandler- aggregateId- command-```--## Database Location--### File-Based Storage-```haskell--- Relative path-withSqlitePool "events.db" 1 $ \pool -> ...---- Absolute path-withSqlitePool "/var/lib/myapp/events.db" 1 $ \pool -> ...---- User-specific location-home <- getHomeDirectory-let dbPath = home </> ".myapp" </> "events.db"-withSqlitePool dbPath 1 $ \pool -> ...-```--### In-Memory Storage (Testing)-```haskell--- Temporary in-memory database-withSqlitePool ":memory:" 1 $ \pool -> ...-```--## Configuration--### Connection Pool-SQLite works best with a single connection per process:--```haskell--- Recommended for SQLite-withSqlitePool "events.db" 1 $ \pool -> ...-```+`eventium-sqlite` provides a lightweight, file-based event store using SQLite.+Ideal for CLI tools, single-process applications, and development environments+where a full database server is unnecessary. -### WAL Mode (Recommended)-Enable Write-Ahead Logging for better concurrent read performance:+## API ```haskell-withSqlitePool "events.db" 1 $ \pool -> do- flip runSqlPool pool $ do- rawExecute "PRAGMA journal_mode=WAL;" []- runMigration migrateAll-```--Benefits:-- Readers don't block writers-- Better performance for read-heavy workloads-- Safer concurrent access--## Performance--Typical SQLite event store performance:-- **Writes**: ~1000-3000 events/sec-- **Reads**: ~5000-20000 events/sec-- **Storage**: ~1KB per event (JSON serialized)--### Optimization Tips--1. **Use WAL Mode** - Better concurrent access-2. **Batch Writes** - Multiple events per transaction-3. **Index Strategy** - Default indexes cover common queries-4. **VACUUM Regularly** - Reclaim space from deleted data-5. **Synchronous Mode** - Balance durability vs speed--## Backup & Recovery--### Simple File Copy-```bash-# Stop application or ensure no writes-cp events.db events.db.backup--# Or use SQLite backup command-sqlite3 events.db ".backup events.db.backup"-```+sqliteEventStoreWriter+ :: (MonadIO m)+ => SqlEventStoreConfig entity serialized+ -> VersionedEventStoreWriter (SqlPersistT m) serialized -### Continuous Backup-```bash-# With WAL mode, backup while app runs-sqlite3 events.db ".backup events.db.backup"+initializeSqliteEventStore+ :: (MonadIO m)+ => SqlEventStoreConfig entity serialized+ -> ConnectionPool+ -> m () ``` -## Migration from In-Memory+Readers come from `eventium-sql-common` (re-exported): ```haskell--- Development: in-memory-development :: IO ()-development = withSqlitePool ":memory:" 1 $ \pool -> ...---- Production: file-based-production :: IO ()-production = withSqlitePool "events.db" 1 $ \pool -> ...+sqlEventStoreReader :: SqlEventStoreConfig entity serialized -> VersionedEventStoreReader (SqlPersistT m) serialized+sqlGlobalEventStoreReader :: SqlEventStoreConfig entity serialized -> GlobalEventStoreReader (SqlPersistT m) serialized ``` -## Example: Complete CLI Application+## Usage ```haskell import Eventium.Store.Sqlite-import System.Directory (getAppUserDataDirectory)+import Database.Persist.Sqlite main :: IO ()-main = do- -- Store in application data directory- dataDir <- getAppUserDataDirectory "myapp"- createDirectoryIfMissing True dataDir- - let dbPath = dataDir </> "events.db"- - withSqlitePool dbPath 1 $ \pool -> do- -- Initialize on first run- flip runSqlPool pool $ do- rawExecute "PRAGMA journal_mode=WAL;" []- runMigration migrateAll- - -- Run application- runApp pool+main = runNoLoggingT $ do+ pool <- createSqlitePool "events.db" 1+ liftIO $ initializeSqliteEventStore defaultSqlEventStoreConfig pool++ let writer = sqliteEventStoreWriter defaultSqlEventStoreConfig+ reader = sqlEventStoreReader defaultSqlEventStoreConfig+ -- writer and reader operate in SqlPersistT m ``` -## Tools+`initializeSqliteEventStore` runs migrations and creates a UUID index. -### SQLite CLI-```bash-# Open database-sqlite3 events.db+## When to Use -# Inspect schema-.schema+| Scenario | Recommendation |+|----------|----------------|+| CLI tools, desktop apps | SQLite |+| Development / prototyping | SQLite or Memory |+| Multi-process / production | PostgreSQL |+| Unit tests (no persistence needed) | Memory | -# Query events-SELECT * FROM events ORDER BY version DESC LIMIT 10;+## Installation -# Check database size-.dbinfo+```yaml+dependencies:+ - eventium-core+ - eventium-sqlite+ - persistent-sqlite ``` -## Limitations--- **Single Writer** - Only one process should write at a time-- **File Locking** - May have issues on network filesystems-- **Database Size** - Practical limit around 100GB-1TB-- **Concurrent Writes** - Limited compared to PostgreSQL- ## Documentation -- [Main README](../README.md) - Project overview-- [SQL Common](../eventium-sql-common/) - Shared SQL utilities-- [Design Documentation](../DESIGN.md) - Architecture details-- [Examples](../examples/) - Bank and cafe examples use SQLite+- [Main README](../README.md)+- [SQL Common](../eventium-sql-common/)+- [Design](../DESIGN.md)+- [Cafe Example](../examples/cafe/) and [Bank Example](../examples/bank/) both use SQLite ## License -MIT - see [LICENSE.md](LICENSE.md)+MIT -- see [LICENSE.md](LICENSE.md)
eventium-sqlite.cabal view
@@ -1,11 +1,11 @@ cabal-version: 1.12 --- This file has been generated from package.yaml by hpack version 0.37.0.+-- This file has been generated from package.yaml by hpack version 0.38.3. -- -- see: https://github.com/sol/hpack name: eventium-sqlite-version: 0.1.0+version: 0.2.1 synopsis: SQLite implementations for eventium description: Eventium-sqlite provides a SQLite-based event store implementation for the Eventium event sourcing framework. It uses the Persistent library for type-safe database access and provides efficient event@@ -28,37 +28,55 @@ type: git location: https://github.com/aleks-sidorenko/eventium +flag ci+ description: Enable -Werror for CI builds+ manual: True+ default: False+ library exposed-modules:+ Eventium.ProjectionCache.Sqlite Eventium.Store.Sqlite other-modules: Paths_eventium_sqlite hs-source-dirs: src- ghc-options: -Wall+ default-extensions:+ NoFieldSelectors+ DuplicateRecordFields+ OverloadedRecordDot+ ghc-options: -Wall -Wcompat -Widentities -Wincomplete-record-updates -Wredundant-constraints -Wno-type-defaults -Wno-incomplete-uni-patterns build-depends: aeson >=1.5 && <2.3 , base >=4.9 && <5 , bytestring >=0.10 && <0.13- , eventium-core- , eventium-sql-common+ , eventium-core >=0.2.0 && <0.3.0+ , eventium-sql-common >=0.2.0 && <0.3.0 , mtl >=2.2 && <2.4- , persistent >=2.13 && <2.15+ , persistent >=2.14 && <2.18 , text >=1.2 && <2.2- , uuid >=1.3 && <1.4+ , uuid ==1.3.* default-language: Haskell2010+ if flag(ci)+ ghc-options: -Werror test-suite spec type: exitcode-stdio-1.0 main-is: Spec.hs other-modules:+ Eventium.ProjectionCache.SqliteSpec Eventium.Store.SqliteSpec+ Eventium.ProjectionCache.Sqlite Eventium.Store.Sqlite Paths_eventium_sqlite hs-source-dirs: tests src- ghc-options: -Wall+ default-extensions:+ NoFieldSelectors+ DuplicateRecordFields+ OverloadedRecordDot+ ghc-options: -Wall -Wcompat -Widentities -Wincomplete-record-updates -Wredundant-constraints -Wno-type-defaults -Wno-incomplete-uni-patterns build-tool-depends: hspec-discover:hspec-discover build-depends:@@ -66,13 +84,15 @@ , aeson >=1.5 && <2.3 , base >=4.9 && <5 , bytestring >=0.10 && <0.13- , eventium-core- , eventium-sql-common- , eventium-test-helpers+ , eventium-core >=0.2.0 && <0.3.0+ , eventium-sql-common >=0.2.0 && <0.3.0+ , eventium-testkit , hspec , mtl >=2.2 && <2.4- , persistent >=2.13 && <2.15+ , persistent >=2.14 && <2.18 , persistent-sqlite , text >=1.2 && <2.2- , uuid >=1.3 && <1.4+ , uuid ==1.3.* default-language: Haskell2010+ if flag(ci)+ ghc-options: -Werror
+ src/Eventium/ProjectionCache/Sqlite.hs view
@@ -0,0 +1,52 @@+module Eventium.ProjectionCache.Sqlite+ ( sqliteVersionedProjectionCache,+ sqliteGlobalProjectionCache,+ sqliteCheckpointStore,+ CheckpointName (..),+ initializeSqliteProjectionCache,+ migrateProjectionSnapshot,+ )+where++import Control.Monad (void)+import Control.Monad.IO.Class (MonadIO, liftIO)+import Database.Persist.Sql (ConnectionPool, SqlPersistT, runMigrationSilent, runSqlPool)+import Eventium.EventSubscription (CheckpointStore)+import Eventium.ProjectionCache.Sql (CheckpointName (..), ProjectionName, migrateProjectionSnapshot, sqlCheckpointStore, sqlGlobalProjectionCache, sqlVersionedProjectionCache)+import Eventium.ProjectionCache.Types (ProjectionCache)+import Eventium.Store.Class (EventVersion, SequenceNumber)+import Eventium.Store.Sql.JSONString (JSONString)+import Eventium.UUID (UUID)++-- | SQLite-backed 'ProjectionCache' for per-entity snapshots.+-- Alias for 'sqlVersionedProjectionCache'.+sqliteVersionedProjectionCache ::+ (MonadIO m) =>+ ProjectionName ->+ ProjectionCache UUID EventVersion JSONString (SqlPersistT m)+sqliteVersionedProjectionCache = sqlVersionedProjectionCache++-- | SQLite-backed 'ProjectionCache' for global blob snapshots.+-- Alias for 'sqlGlobalProjectionCache'.+sqliteGlobalProjectionCache ::+ (MonadIO m) =>+ ProjectionName ->+ ProjectionCache () SequenceNumber JSONString (SqlPersistT m)+sqliteGlobalProjectionCache = sqlGlobalProjectionCache++-- | SQLite-backed 'CheckpointStore' for tracking subscription position.+-- Alias for 'sqlCheckpointStore'.+sqliteCheckpointStore ::+ (MonadIO m) =>+ CheckpointName ->+ CheckpointStore (SqlPersistT m) SequenceNumber+sqliteCheckpointStore = sqlCheckpointStore++-- | Run migrations to create the projection_snapshots table in SQLite.+-- Mirrors 'initializeSqliteEventStore' from "Eventium.Store.Sqlite".+initializeSqliteProjectionCache ::+ (MonadIO m) =>+ ConnectionPool ->+ m ()+initializeSqliteProjectionCache pool =+ liftIO $ void $ runSqlPool (runMigrationSilent migrateProjectionSnapshot) pool
src/Eventium/Store/Sqlite.hs view
@@ -1,11 +1,11 @@ {-# LANGUAGE OverloadedStrings #-}-{-# LANGUAGE RecordWildCards #-} {-# LANGUAGE TypeFamilies #-} {-# LANGUAGE TypeOperators #-} -- | Defines an Sqlite event store. module Eventium.Store.Sqlite ( sqliteEventStoreWriter,+ sqliteEventStoreWriterTagged, initializeSqliteEventStore, module Eventium.Store.Class, module Eventium.Store.Sql,@@ -15,8 +15,6 @@ import Control.Monad.Reader import Data.Text (Text) import Database.Persist-import Database.Persist.Class (SafeToInsert)-import Database.Persist.Names (EntityNameDB (..), FieldNameDB (..)) import Database.Persist.Sql import Eventium.Store.Class import Eventium.Store.Sql@@ -32,6 +30,17 @@ getLatestVersion = sqlMaxEventVersion config maxSqliteVersionSql storeEvents' = sqlStoreEvents config Nothing maxSqliteVersionSql +-- | Like 'sqliteEventStoreWriter' but accepts 'TaggedEvent's,+-- preserving the metadata attached to each event.+sqliteEventStoreWriterTagged ::+ (MonadIO m, PersistEntity entity, PersistEntityBackend entity ~ SqlBackend, SafeToInsert entity) =>+ SqlEventStoreConfig entity serialized ->+ VersionedEventStoreWriter (SqlPersistT m) (TaggedEvent serialized)+sqliteEventStoreWriterTagged config = EventStoreWriter $ transactionalExpectedWriteHelper getLatestVersion storeEvents'+ where+ getLatestVersion = sqlMaxEventVersion config maxSqliteVersionSql+ storeEvents' = sqlStoreEventsTagged config Nothing maxSqliteVersionSql+ maxSqliteVersionSql :: FieldNameDB -> FieldNameDB -> FieldNameDB -> Text maxSqliteVersionSql (FieldNameDB tableName) (FieldNameDB uuidFieldName) (FieldNameDB versionFieldName) = "SELECT IFNULL(MAX(" <> versionFieldName <> "), -1) FROM " <> tableName <> " WHERE " <> uuidFieldName <> " = ?"@@ -39,17 +48,17 @@ -- | This functions runs the migrations required to create the events table and -- also adds an index on the UUID column. initializeSqliteEventStore ::- (MonadIO m, PersistEntity entity, PersistEntityBackend entity ~ SqlBackend) =>+ (MonadIO m, PersistEntity entity) => SqlEventStoreConfig entity serialized -> ConnectionPool -> m ()-initializeSqliteEventStore SqlEventStoreConfig {..} pool = do+initializeSqliteEventStore config pool = do -- Run migrations _ <- liftIO $ runSqlPool (runMigrationSilent migrateSqlEvent) pool -- Create index on uuid field so retrieval is very fast- let tableName = unEntityNameDB $ tableDBName (sqlEventStoreConfigSequenceMakeEntity undefined undefined undefined)- uuidFieldName = unFieldNameDB $ fieldDBName sqlEventStoreConfigSequenceNumberField+ let tableName = unEntityNameDB $ tableDBName (config.sequenceMakeEntity undefined undefined undefined undefined)+ uuidFieldName = unFieldNameDB $ fieldDBName config.sequenceNumberField indexSql = "CREATE INDEX IF NOT EXISTS " <> uuidFieldName
+ tests/Eventium/ProjectionCache/SqliteSpec.hs view
@@ -0,0 +1,59 @@+{-# LANGUAGE OverloadedStrings #-}++module Eventium.ProjectionCache.SqliteSpec (spec) where++import Database.Persist.Sqlite+import Eventium.ProjectionCache.Cache (codecProjectionCache)+import Eventium.ProjectionCache.Sqlite+import Eventium.Store.Sqlite+import Eventium.Testkit+import Test.Hspec++spec :: Spec+spec = do+ describe "SQLite versioned projection cache" $ do+ versionedProjectionCacheSpec sqliteVersionedProjectionCacheRunner++ describe "SQLite global projection cache" $ do+ globalProjectionCacheSpec sqliteGlobalProjectionCacheRunner++ describe "SQLite checkpoint store" $ do+ checkpointStoreSpec sqliteCheckpointStoreRunner++makeStore ::+ IO+ ( VersionedEventStoreWriter (SqlPersistT IO) CounterEvent,+ VersionedEventStoreReader (SqlPersistT IO) CounterEvent,+ ConnectionPool+ )+makeStore = do+ pool <- liftIO $ runNoLoggingT (createSqlitePool ":memory:" 1)+ let writer = codecEventStoreWriter jsonStringCodec $ sqliteEventStoreWriter defaultSqlEventStoreConfig+ reader = codecVersionedEventStoreReader jsonStringCodec $ sqlEventStoreReader defaultSqlEventStoreConfig+ initializeSqliteEventStore defaultSqlEventStoreConfig pool+ initializeSqliteProjectionCache pool+ return (writer, reader, pool)++sqliteVersionedProjectionCacheRunner :: VersionedProjectionCacheRunner (SqlPersistT IO)+sqliteVersionedProjectionCacheRunner = VersionedProjectionCacheRunner $ \action -> do+ (writer, reader, pool) <- makeStore+ let cache =+ codecProjectionCache jsonStringCodec $+ sqliteVersionedProjectionCache (ProjectionName "test_versioned")+ runSqlPool (action writer reader cache) pool++sqliteGlobalProjectionCacheRunner :: GlobalProjectionCacheRunner (SqlPersistT IO)+sqliteGlobalProjectionCacheRunner = GlobalProjectionCacheRunner $ \action -> do+ (writer, _, pool) <- makeStore+ let globalReader =+ codecGlobalEventStoreReader jsonStringCodec $+ sqlGlobalEventStoreReader defaultSqlEventStoreConfig+ cache =+ codecProjectionCache jsonStringCodec $+ sqliteGlobalProjectionCache (ProjectionName "test_global")+ runSqlPool (action writer globalReader cache) pool++sqliteCheckpointStoreRunner :: CheckpointStoreRunner (SqlPersistT IO)+sqliteCheckpointStoreRunner = CheckpointStoreRunner $ \action -> do+ (_, _, pool) <- makeStore+ runSqlPool (action (sqliteCheckpointStore (CheckpointName "test_checkpoint"))) pool
tests/Eventium/Store/SqliteSpec.hs view
@@ -4,7 +4,7 @@ import Database.Persist.Sqlite import Eventium.Store.Sqlite-import Eventium.TestHelpers+import Eventium.Testkit import Test.Hspec spec :: Spec@@ -22,8 +22,8 @@ makeStore :: IO (VersionedEventStoreWriter (SqlPersistT IO) CounterEvent, VersionedEventStoreReader (SqlPersistT IO) CounterEvent, ConnectionPool) makeStore = do pool <- liftIO $ runNoLoggingT (createSqlitePool ":memory:" 1)- let writer = serializedEventStoreWriter jsonStringSerializer $ sqliteEventStoreWriter defaultSqlEventStoreConfig- reader = serializedVersionedEventStoreReader jsonStringSerializer $ sqlEventStoreReader defaultSqlEventStoreConfig+ let writer = codecEventStoreWriter jsonStringCodec $ sqliteEventStoreWriter defaultSqlEventStoreConfig+ reader = codecVersionedEventStoreReader jsonStringCodec $ sqlEventStoreReader defaultSqlEventStoreConfig initializeSqliteEventStore defaultSqlEventStoreConfig pool return (writer, reader, pool) @@ -35,7 +35,7 @@ sqliteStoreGlobalRunner :: GlobalStreamEventStoreRunner (SqlPersistT IO) sqliteStoreGlobalRunner = GlobalStreamEventStoreRunner $ \action -> do (writer, _, pool) <- makeStore- let globalStore = serializedGlobalEventStoreReader jsonStringSerializer (sqlGlobalEventStoreReader defaultSqlEventStoreConfig)+ let globalStore = codecGlobalEventStoreReader jsonStringCodec (sqlGlobalEventStoreReader defaultSqlEventStoreConfig) runSqlPool (action writer globalStore) pool makeIOStore :: IO (VersionedEventStoreWriter IO CounterEvent, VersionedEventStoreReader IO CounterEvent, ConnectionPool)@@ -53,6 +53,6 @@ sqliteIOStoreGlobalRunner :: GlobalStreamEventStoreRunner IO sqliteIOStoreGlobalRunner = GlobalStreamEventStoreRunner $ \action -> do (writer, _, pool) <- makeIOStore- let globalStore = serializedGlobalEventStoreReader jsonStringSerializer (sqlGlobalEventStoreReader defaultSqlEventStoreConfig)+ let globalStore = codecGlobalEventStoreReader jsonStringCodec (sqlGlobalEventStoreReader defaultSqlEventStoreConfig) globalStore' = runEventStoreReaderUsing (`runSqlPool` pool) globalStore action writer globalStore'