hlrdb 0.2.0.1 → 0.3.0.0

raw patch · 5 files changed

+169/−21 lines, 5 filesdep ~bytestringdep ~cryptonitedep ~memorysetup-changedPVP ok

version bump matches the API change (PVP)

Dependency ranges changed: bytestring, cryptonite, memory, store

API changes (from Hackage documentation)

Files

• CHANGELOG.md +9/−0
• README.md +138/−0
• Setup.hs +0/−2
• hlrdb.cabal +12/−9
• src/HLRDB.hs +10/−10

@@ -0,0 +1,9 @@+# New in 0.3+- Generalized path declarations to any `Store` from `IsIdentifier`.++# New in 0.2++- Core dependency bumped to 0.1.1+- Commands related to key expiration have been moved to the core module, assigned their corresponding names in Redis, and depend on types from the `time` package.+- `time-exts` has been dropped in favor of `time` for handling timestamps in identifiers, though the semantics should be the same and data should be backwards compatible.+

@@ -0,0 +1,138 @@+# High-level Redis Database++HLRDB is an opinionated, high-level, type-driven library for modeling Redis-backed database architecture.++This package provides an easy API for you to declare your data paths in Redis, but in doing so makes many decisions for you about how to serialize and deserialize values, construct identifiers, and define path names. If you want more control over these aspects, you may instead use the HLRDB Core package, which simply defines the commands and the abstract API without opining on these matters.++## Overview++Redis is a hash table database with several builtin primitive data structures. It does not use SQL, but instead uses [its own system of primitive commands](https://redis.io/commands). You may find primitive Haskell bindings for these commands [in the Hedis library](https://hackage.haskell.org/package/hedis), on which this library depends. HLRDB provides a type-driven, high-level abstraction on top of this.++```haskell+-- minimal end-to-end, runnable example+import Data.Store+import Database.Redis (checkedConnect,defaultConnectInfo,runRedis)+import HLRDB++newtype CommentId = CommentId Identifier deriving (Eq,Ord,Show,Store,IsIdentifier)+newtype Comment = Comment String deriving (Eq,Ord,Show,Store)++cidToComment :: RedisBasic CommentId (Maybe Comment)+cidToComment = declareBasic "canonical mapping from CommentId to Comment"++main :: IO ()+main = do+  -- connect to Redis+  rconn <- checkedConnect defaultConnectInfo++  cid :: CommentId <- genId++  c :: Maybe Comment <- runRedis rconn $ do+    -- create a comment+    set' cidToComment cid $ Comment "hi"+    -- read it back+    get cidToComment cid++  print c++```++## Identifiers++Use newtypes for `Identifier` for your various data types:++```haskell++newtype CommentId = CommentId Identifier deriving (Eq,Ord,Show,Store,IsIdentifier)++-- use genId to create new identifiers:+example :: IO CommentId+example = genId+```++## Data structures++Redis structures are mostly indexed by two types: their identifier and their value. When you declare a structure, you need to provide a unique description, which serves two purposes: first, it helps document what the purpose of the path is, and second, the hash of this string is how HLRDB distinguishes between multiple paths of the same type.++### Basic++```haskell+-- RedisBasic is used when for standard key-value storage.+cidToComment :: RedisBasic CommentId (Maybe Comment)+cidToComment = declareBasic "canonical mapping from CommentId to Comment"++-- RedisIntegral will treat a non-existent value as 0+cidToScore :: RedisIntegral CommentId Integer+cidToScore = declareIntegral "comment score"++-- Use `declareBasicZero` to choose your own "zero" for the data type+threadIdToComments :: RedisBasic ThreadId (RoseTree CommentId)+threadIdToComments = declareBasicZero "reddit-style comment threads" Empty+```++### Other Redis structures++For lists and sorted sets, you may optionally provide a `TrimScheme` (a record with two fields, `softCardinality :: Integer` and `trimProbability :: Double`). When provided, HLRDB will automatically trim the structures in Redis to their proper size whenever data is added.++```haskell+-- hset, basically a sub-hash table with a few extra primitive commands+voteHSet :: RedisStructure (HSET CommentId) UserId Vote+voteHSet = declareHSet "whether a user has voted a comment up or down"++-- list, with automatic max-length management with TrimScheme+tidToComments :: RedisList ThreadId CommentId+tidToComments = declareList "non-recursive comment threads" $ Just $ TrimScheme 1000 0.1++-- sorted sets store items by golf score - lower is better. supports TrimScheme+popularItems :: RedisSSet UserId PostId+popularItems = declareSSet "popular content" $ Just $ TrimScheme 1000 0.01 -- 1k max; trim with probability 0.01++-- set is intuitive+blockedUsers :: RedisSet UserId UserId+blockedUsers = declareSet "a user's block list"++```++### Global paths++You may use the global variants of the above to declare paths indexed simply on `()`, rather than an `Identifier` newtype:++```haskell+bannedUsers :: RedisSet () UserId+bannedUsers = declareGlobalSet "global ban list"+```++Once you've declared any of the above structures, you may use the Redis monad to perform operations on them. You may find the operations available for each structure defined in the [HLRDB/Structures](https://github.com/identicalsnowflake/hlrdb-core/tree/master/src/HLRDB/Structures) folder (found in hlrdb-core) for that particular structure. The commands are similar to the original Redis API, but have been cleaned up and re-imagined to support more of a Haskell dialect (e.g., list commands do not crash when passed `[]` as they do in Redis).++## Lookup Aggregation++You may lift `RedisBasic i v` (and `RedisIntegral i v`, which is a subtype) paths to `i ⟿ v` queries, which can be combined together in several ways, resulting in a single `mget` command being executed in Redis. This allows constructing detailed data views in an efficient manner.++If you prefer, `Query i v` is a non-infix alias for `i ⟿ v`. You may also use the ASCII version, `~~>`.++```haskell++newtype Views = Views Integer deriving (Show,Eq,Ord,Num,Enum,Real,Integral)+newtype Likes = Likes Integer deriving (Show,Eq,Ord,Num,Enum,Real,Integral)+++cidToViews :: RedisIntegral CommentId Views+cidToViews = declareIntegral "comment views"++cidToLikes :: RedisIntegral CommentId Likes+cidToLikes = declareIntegral "comment likes"+++queryBoth :: CommentId ⟿ (Views , Likes)+queryBoth = (,) <$> liftq cidToViews <*> liftq cidToLikes++reifyToRedis :: CommentId -> Redis (Views , Likes)+reifyToRedis = mget queryBoth++```++I've written an [in-depth article discussing aggregation here](https://identicalsnowflake.github.io/QueryAggregation.html), but the two most important takeaways are that `⟿` is a `Traversing` `Profunctor` and an `Applicative`.++## Demo++There is a [simple demo repository](https://github.com/identicalsnowflake/hlrdb-demo) demonstrating the end-to-end process of defining data models and performing read/write actions.

@@ -1,2 +0,0 @@-import Distribution.Simple-main = defaultMain

@@ -1,5 +1,5 @@ name:                hlrdb-version:             0.2.0.1+version:             0.3.0.0 synopsis:            High-level Redis Database description:         A library for type-driven interaction with Redis license:             MIT@@ -11,6 +11,9 @@ cabal-version:       2.0 homepage:            https://github.com/identicalsnowflake/hlrdb bug-reports:         https://github.com/identicalsnowflake/hlrdb/issues+extra-source-files:+  CHANGELOG.md+  README.md  source-repository head   type:     git@@ -22,15 +25,15 @@       base >= 4.9 && < 5.0     , base64-bytestring ^>= 1.0.0.1     , bytestring ^>= 0.10.8.1-    , cryptonite (>= 0.24 && < 0.25) || ^>= 0.25-    , hashable ^>= 1.2.6.1-    , hedis ^>= 0.10.1-    , hlrdb-core ^>= 0.1.1-    , memory ^>= 0.14.8+    , cryptonite (>= 0.24 && < 0.26) || ^>= 0.26+    , hashable+    , hedis+    , hlrdb-core >= 0.1.1 && < 0.2+    , memory (>= 0.14.8 && <=0.14.18) || ^>= 0.14.18     , random ^>= 1.1-    , store ^>= 0.5.0-    , time (>=1.6 && <1.9.1) || ^>= 1.9.1-    , unordered-containers ^>= 0.2.8.0+    , store ^>= 0.5.1.1+    , time+    , unordered-containers   hs-source-dirs:      src   default-language:    Haskell2010   default-extensions:

@@ -187,9 +187,9 @@     . fromString  -- | If for some reason you need the actual, raw key name (which you may use with the low-level commands in hedis), you may obtain it via @encodePath@.-encodePath :: IsIdentifier a => PathName -> a -> ByteString+encodePath :: Store a => PathName -> a -> ByteString encodePath (PathName n) =-  (<>) n . encode . toIdentifier+  (<>) n . encode  failDecode :: PeekException -> a failDecode e = error $ "Unexpected data encoding from Redis: " <> show e@@ -211,7 +211,7 @@ -- @ --  {-# INLINE declareBasic #-}-declareBasic :: (IsIdentifier i, Store v) => PathName -> RedisBasic i (Maybe v)+declareBasic :: (Store i, Store v) => PathName -> RedisBasic i (Maybe v) declareBasic pathName = RKeyValue $   E (encodePath pathName)     (fmap encode)@@ -222,13 +222,13 @@  -- | Standard key-value store, but backed by a primitive integer in Redis, enabling extra commands like @incr@ {-# INLINE declareIntegral #-}-declareIntegral :: (IsIdentifier i, Integral b) => PathName -> RedisIntegral i b+declareIntegral :: (Store i, Integral b) => PathName -> RedisIntegral i b declareIntegral p =   RKeyValueInteger (encodePath p) toInteger fromIntegral  -- | Allows defining your own "zero" value. An example might be RoseTree, where a non-existant value in Redis can be mapped to a sensible empty value in Haskell. {-# INLINE declareBasicZero #-}-declareBasicZero :: (IsIdentifier i, Store v) => PathName -> v -> RedisBasic i v+declareBasicZero :: (Store i, Store v) => PathName -> v -> RedisBasic i v declareBasicZero pathName zero = RKeyValue $   E (encodePath pathName)     (Just . encode)@@ -240,24 +240,24 @@  -- | Standard Redis list, supporting prepends, appends, and range access. If a @TrimScheme@ is provided, operations will automatically trim the list to the specified length. {-# INLINE declareList #-}-declareList :: (IsIdentifier i, Store v) => PathName -> Maybe TrimScheme -> RedisList i v+declareList :: (Store i, Store v) => PathName -> Maybe TrimScheme -> RedisList i v declareList pathName = RList $ E (encodePath pathName) (pure . encode) (decode' . runIdentity)  -- | A sub-hash table, using the sub-index type @s@. @s@ here is only required to be Storable rather than IsIdentifier, but in practice you'll probably use identifiers for @s@, too. {-# INLINE declareHSet #-}-declareHSet :: (IsIdentifier i, Store s, Store v) => PathName -> RedisHSet i s v+declareHSet :: (Store i, Store s, Store v) => PathName -> RedisHSet i s v declareHSet pathName =   RHSet (E (encodePath pathName) (pure . encode) (decode' . runIdentity)) (HSET encode decode')  -- | A set in Redis. {-# INLINE declareSet #-}-declareSet :: (IsIdentifier i, Store v) => PathName -> RedisSet i v+declareSet :: (Store i, Store v) => PathName -> RedisSet i v declareSet pathName =   RSet $ E (encodePath pathName) (pure . encode) (decode' . runIdentity)  -- | A sorted set in Redis. You may optionally provide a trim scheme, which will automatically manage the sorted set's size for you. {-# INLINE declareSSet #-}-declareSSet :: (IsIdentifier i, Store v) => PathName -> Maybe TrimScheme -> RedisSSet i v+declareSSet :: (Store i, Store v) => PathName -> Maybe TrimScheme -> RedisSSet i v declareSSet pathName =   RSortedSet $ E (encodePath pathName) (pure . encode) (decode' . runIdentity) @@ -332,7 +332,7 @@         zeroIdentifier :: (IsIdentifier i) => i         zeroIdentifier = fromIdentifier $ Identifier (0,0,0,0) --- | Note that despite the pretty type signature, the actual implementation of @foldPath@ in Redis is slow (it uses the global scan command, so its run time is proportional to the number of total keys in Redis, *not* the number of keys specifically related to the given path). You should only use @foldPath@ for administrative tasks, and never for any public API.+-- | Note that despite the pretty type signature, the actual implementation of @foldPath@ in Redis is slow (it uses the global scan command, so its run time is proportional to the number of total keys in Redis, *not* the number of keys specifically related to the given path). You should only use @foldPath@ for administrative tasks, and never for any public API. Further, this method is only guaranteed to work if you've declared your @RedisStructure@s using the declarative tools in this module: if you declared a path yourself, please ensure it is compatible with the pathing convention in this module (namely, a 5-byte prefix).  foldPath :: (MonadRedis m , IsIdentifier i , Store v) => RedisStructure s i v -> (a -> i -> m a) -> a -> m a foldPath p f z = go (cursor0,z)