valiant-streamly (empty) → 0.1.0.0

raw patch · 7 files changed

+329/−0 lines, 7 filesdep +basedep +bytestringdep +hspec

Dependencies added: base, bytestring, hspec, pg-wire, streamly-core, text, valiant, valiant-streamly

Files

CHANGELOG.md +14/−0
LICENSE +27/−0
README.md +35/−0
src/Valiant/Streamly.hs +56/−0
test/Main.hs +58/−0
test/TestSupport.hs +63/−0
valiant-streamly.cabal +76/−0

+ CHANGELOG.md view

@@ -0,0 +1,14 @@+# Changelog++The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).++## [0.1.0.0] - 2026-04-25++Initial release. Streamly adapter for+[`valiant`](https://hackage.haskell.org/package/valiant). Exposes+`selectStreamly` (cursor-based) and `foldStreamly` (single-shot+extended-protocol) as `Stream IO r` values from+[`streamly-core`](https://hackage.haskell.org/package/streamly-core).++[0.1.0.0]: https://github.com/joshburgess/valiant/releases/tag/v0.1.0.0

+ LICENSE view

@@ -0,0 +1,27 @@+Copyright (c) 2026 Josh Burgess++Redistribution and use in source and binary forms, with or without+modification, are permitted provided that the following conditions are met:++1. Redistributions of source code must retain the above copyright notice,+   this list of conditions and the following disclaimer.++2. Redistributions in binary form must reproduce the above copyright notice,+   this list of conditions and the following disclaimer in the documentation+   and/or other materials provided with the distribution.++3. Neither the name of the copyright holder nor the names of its+   contributors may be used to endorse or promote products derived from this+   software without specific prior written permission.++THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE+ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE+LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF+SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS+INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN+CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)+ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE+POSSIBILITY OF SUCH DAMAGE.

+ README.md view

@@ -0,0 +1,35 @@+# valiant-streamly++[Streamly](https://hackage.haskell.org/package/streamly-core) adapter+for [`valiant`](https://hackage.haskell.org/package/valiant).++Produces `Stream IO r` values from query results.++## Quick start++```haskell+import Valiant+import Valiant.Streamly+import Streamly.Data.Stream qualified as Stream+import Streamly.Data.Fold qualified as Fold++countUsers :: Pool -> IO Int+countUsers pool =+  withTransaction pool $ \tx ->+    Stream.fold Fold.length $+      selectStreamly (txConn tx) listAllUsers () 500+```++## Two streaming strategies++- `selectStreamly conn stmt params batchSize` — cursor-based. Must run+  inside a transaction.+- `foldStreamly conn stmt params` — single-shot extended-protocol+  query. No transaction required.++Both functions accumulate one result set at a time before yielding.+For truly incremental memory use, drive `Valiant.withCursor` /+`Valiant.fetchBatch` directly.++See the [valiant tutorial](https://github.com/joshburgess/valiant/blob/main/docs/TUTORIAL.md)+for `Statement` definitions and the `valiant prepare` workflow.

+ src/Valiant/Streamly.hs view

@@ -0,0 +1,56 @@+{-# OPTIONS_GHC -fno-full-laziness #-}++-- | Streamly streaming adapter for valiant.+--+-- Produces @Stream IO r@ values from query results, enabling+-- high-performance composable stream processing with streamly.+--+-- @+-- import Valiant+-- import Valiant.Streamly+-- import Streamly.Data.Stream qualified as Stream+-- import Streamly.Data.Fold qualified as Fold+--+-- withTransaction pool $ \\tx -> do+--   count <- Stream.fold Fold.length $+--     selectStreamly (txConn tx) listAllUsers () 500+--   print count+-- @+module Valiant.Streamly+  ( -- * Cursor-based streaming (requires transaction)+    selectStreamly+    -- * Fold-based streaming (no transaction required)+  , foldStreamly+  ) where++import Streamly.Data.Stream (Stream)+import Streamly.Data.Stream qualified as Stream+import Valiant (Connection, Statement, fetchAll, fetchAllCursor)++-- | Stream query results using a server-side cursor.+--+-- Must be called inside a transaction. Fetches rows in batches and+-- yields decoded rows one at a time.+--+-- Note: rows are accumulated in memory before yielding (see+-- 'Valiant.Conduit.selectSource' for the same caveat).+selectStreamly+  :: Connection+  -> Statement p r+  -> p+  -> Int+  -- ^ Batch size+  -> Stream IO r+selectStreamly conn stmt params batchSize =+  Stream.concatEffect (Stream.fromList <$> fetchAllCursor conn stmt params batchSize)++-- | Stream query results using single-shot extended-protocol execution.+--+-- Does not require a transaction.+foldStreamly+  :: Connection+  -> Statement p r+  -> p+  -> Stream IO r+foldStreamly conn stmt params =+  Stream.concatEffect (Stream.fromList <$> fetchAll conn stmt params)

+ test/Main.hs view

@@ -0,0 +1,58 @@+module Main where++import Data.Int (Int32)+import Data.Text (Text)+import Valiant+import Valiant.Streamly (foldStreamly, selectStreamly)+import Streamly.Data.Fold qualified as Fold+import Streamly.Data.Stream qualified as Stream+import Test.Hspec+import TestSupport++main :: IO ()+main = hspec $ do+  describe "Valiant.Streamly" $ do+    describe "selectStreamly (cursor-based)" $ do+      it "streams all rows from a cursor" $ do+        withTestConnection $ \conn -> withSchema conn $ do+          insertTestUsers conn+          withTransactionConn conn $ \tx -> do+            rows <- Stream.fold Fold.toList $+              selectStreamly (txConn tx) stmtListAll () 2+            length rows `shouldBe` 5++      it "returns empty for no rows" $ do+        withTestConnection $ \conn -> withSchema conn $ do+          withTransactionConn conn $ \tx -> do+            rows <- Stream.fold Fold.toList $+              selectStreamly (txConn tx) stmtListAll () 10+            rows `shouldBe` ([] :: [(Int32, Text)])++      it "works with small batch size" $ do+        withTestConnection $ \conn -> withSchema conn $ do+          insertTestUsers conn+          withTransactionConn conn $ \tx -> do+            rows <- Stream.fold Fold.toList $+              selectStreamly (txConn tx) stmtListAll () 1+            length rows `shouldBe` 5++    describe "foldStreamly (no transaction)" $ do+      it "streams all rows without a transaction" $ do+        withTestConnection $ \conn -> withSchema conn $ do+          insertTestUsers conn+          rows <- Stream.fold Fold.toList $+            foldStreamly conn stmtListAll ()+          length rows `shouldBe` 5++      it "returns empty for no rows" $ do+        withTestConnection $ \conn -> withSchema conn $ do+          rows <- Stream.fold Fold.toList $+            foldStreamly conn stmtListAll ()+          rows `shouldBe` ([] :: [(Int32, Text)])++      it "composes with streamly combinators" $ do+        withTestConnection $ \conn -> withSchema conn $ do+          insertTestUsers conn+          count <- Stream.fold Fold.length $+            foldStreamly conn stmtListAll ()+          count `shouldBe` 5

+ test/TestSupport.hs view

@@ -0,0 +1,63 @@+module TestSupport+  ( withTestConnection+  , withSchema+  , insertTestUsers+  , stmtListAll+  , stmtSelectOne+  ) where++import Data.ByteString (ByteString)+import Data.ByteString.Char8 qualified as BS8+import Data.Int (Int32)+import Data.Text (Text)+import Valiant+import System.Environment (lookupEnv)++requireDatabaseUrl :: IO ByteString+requireDatabaseUrl = do+  mUrl <- lookupEnv "DATABASE_URL"+  case mUrl of+    Just url -> pure (BS8.pack url)+    Nothing -> error "DATABASE_URL is not set."++withTestConnection :: (Connection -> IO a) -> IO a+withTestConnection action = do+  url <- requireDatabaseUrl+  conn <- connectString url+  result <- action conn+  close conn+  pure result++withSchema :: Connection -> IO a -> IO a+withSchema conn action = do+  _ <- simpleQuery conn "DROP TABLE IF EXISTS users_streamly CASCADE"+  _ <- simpleQuery conn+    "CREATE TABLE users_streamly (\+    \  id SERIAL PRIMARY KEY,\+    \  name TEXT NOT NULL,\+    \  email TEXT\+    \)"+  result <- action+  _ <- simpleQuery conn "DROP TABLE IF EXISTS users_streamly CASCADE"+  pure result++insertTestUsers :: Connection -> IO ()+insertTestUsers conn = do+  _ <- simpleQuery conn+    "INSERT INTO users_streamly (name, email) VALUES \+    \('Alice', 'alice@example.com'),\+    \('Bob', 'bob@example.com'),\+    \('Carol', NULL),\+    \('Dave', 'dave@example.com'),\+    \('Eve', 'eve@example.com')"+  pure ()++stmtListAll :: Statement () (Int32, Text)+stmtListAll = mkStatement+  "SELECT id, name FROM users_streamly ORDER BY id"+  [] ["id", "name"] "<test>"++stmtSelectOne :: Statement Int32 (Int32, Text, Maybe Text)+stmtSelectOne = mkStatement+  "SELECT id, name, email FROM users_streamly WHERE id = $1"+  [23] ["id", "name", "email"] "<test>"

+ valiant-streamly.cabal view

@@ -0,0 +1,76 @@+cabal-version:   3.0+name:            valiant-streamly+version:         0.1.0.0+synopsis:        Streamly streaming adapter for valiant+description:+  Stream PostgreSQL query results using the @streamly@ library.+  Provides cursor-based and fold-based producers as @Stream IO r@+  values, enabling high-performance composable stream processing.+license:         BSD-3-Clause+license-file:    LICENSE+author:          Josh Burgess+maintainer:      joshburgess.webdev@gmail.com+category:        Database+homepage:        https://github.com/joshburgess/valiant+bug-reports:     https://github.com/joshburgess/valiant/issues+build-type:      Simple+extra-doc-files:+  README.md+  CHANGELOG.md+tested-with:     GHC ==9.10.3++source-repository head+  type:     git+  location: https://github.com/joshburgess/valiant+  subdir:   adapters/valiant-streamly++flag werror+  description: Enable -Werror for development builds.+  default:     False+  manual:      True++common warnings+  ghc-options: -Wall -Wcompat -Wno-unticked-promoted-constructors -funbox-strict-fields -fspecialise-aggressively+  if flag(werror)+    ghc-options: -Werror++library+  import:           warnings+  hs-source-dirs:   src+  default-language: GHC2021+  default-extensions:+    DerivingStrategies+    OverloadedStrings+    StrictData++  exposed-modules:+    Valiant.Streamly++  build-depends:+    , base            >=4.17 && <5+    , valiant         >=0.1  && <0.2+    , streamly-core   >=0.2  && <0.4++test-suite valiant-streamly-test+  import:           warnings+  type:             exitcode-stdio-1.0+  hs-source-dirs:   test+  main-is:          Main.hs+  default-language: GHC2021+  default-extensions:+    OverloadedStrings++  ghc-options: -rtsopts "-with-rtsopts=-K8K"++  other-modules:+    TestSupport++  build-depends:+    , base            >=4.17 && <5+    , bytestring      >=0.11 && <0.13+    , hspec           >=2.11 && <2.13+    , valiant+    , valiant-streamly+    , pg-wire+    , streamly-core   >=0.2  && <0.4+    , text            >=2.0  && <2.2