valiant-streaming (empty) → 0.1.0.0

raw patch · 7 files changed

+330/−0 lines, 7 filesdep +basedep +bytestringdep +hspec

Dependencies added: base, bytestring, hspec, pg-wire, streaming, text, valiant, valiant-streaming

Files

CHANGELOG.md +14/−0
LICENSE +27/−0
README.md +33/−0
src/Valiant/Stream.hs +58/−0
test/Main.hs +59/−0
test/TestSupport.hs +63/−0
valiant-streaming.cabal +76/−0

+ CHANGELOG.md view

@@ -0,0 +1,14 @@+# Changelog++The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).++## [0.1.0.0] - 2026-04-25++Initial release. Streaming adapter for+[`valiant`](https://hackage.haskell.org/package/valiant) using the+[`streaming`](https://hackage.haskell.org/package/streaming) library.+Exposes `selectStream` (cursor-based) and `foldStream`+(single-shot extended-protocol) as `Stream (Of r) IO ()` values.++[0.1.0.0]: https://github.com/joshburgess/valiant/releases/tag/v0.1.0.0

+ LICENSE view

@@ -0,0 +1,27 @@+Copyright (c) 2026 Josh Burgess++Redistribution and use in source and binary forms, with or without+modification, are permitted provided that the following conditions are met:++1. Redistributions of source code must retain the above copyright notice,+   this list of conditions and the following disclaimer.++2. Redistributions in binary form must reproduce the above copyright notice,+   this list of conditions and the following disclaimer in the documentation+   and/or other materials provided with the distribution.++3. Neither the name of the copyright holder nor the names of its+   contributors may be used to endorse or promote products derived from this+   software without specific prior written permission.++THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE+ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE+LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF+SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS+INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN+CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)+ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE+POSSIBILITY OF SUCH DAMAGE.

+ README.md view

@@ -0,0 +1,33 @@+# valiant-streaming++[`streaming`](https://hackage.haskell.org/package/streaming) adapter+for [`valiant`](https://hackage.haskell.org/package/valiant).++Produces `Stream (Of r) IO ()` values from query results.++## Quick start++```haskell+import Valiant+import Valiant.Stream+import Streaming.Prelude qualified as S++countUsers :: Pool -> IO Int+countUsers pool =+  withTransaction pool $ \tx ->+    S.length_ $ selectStream (txConn tx) listAllUsers () 500+```++## Two streaming strategies++- `selectStream conn stmt params batchSize` — cursor-based. Must run+  inside a transaction.+- `foldStream conn stmt params` — single-shot extended-protocol query.+  No transaction required.++Both functions accumulate one result set at a time before yielding.+For truly incremental memory use, drive `Valiant.withCursor` /+`Valiant.fetchBatch` directly.++See the [valiant tutorial](https://github.com/joshburgess/valiant/blob/main/docs/TUTORIAL.md)+for `Statement` definitions and the `valiant prepare` workflow.

+ src/Valiant/Stream.hs view

@@ -0,0 +1,58 @@+{-# OPTIONS_GHC -fno-full-laziness #-}++-- | Streaming adapter for valiant using the @streaming@ library.+--+-- Produces @Stream (Of r) IO ()@ values from query results, enabling+-- composable stream processing.+--+-- @+-- import Valiant+-- import Valiant.Stream+-- import Streaming.Prelude qualified as S+--+-- withTransaction pool $ \\tx -> do+--   count <- S.length_ $+--     selectStream (txConn tx) listAllUsers () 500+--   print count+-- @+module Valiant.Stream+  ( -- * Cursor-based streaming (requires transaction)+    selectStream+    -- * Fold-based streaming (no transaction required)+  , foldStream+  ) where++import Control.Monad.IO.Class (liftIO)+import Streaming (Of, Stream)+import Streaming.Prelude qualified as S+import Valiant (Connection, Statement, fetchAll, fetchAllCursor)++-- | Stream query results using a server-side cursor.+--+-- Must be called inside a transaction. Fetches rows in batches and+-- yields decoded rows one at a time.+--+-- Note: rows are accumulated in memory before yielding (see+-- 'Valiant.Conduit.selectSource' for the same caveat).+selectStream+  :: Connection+  -> Statement p r+  -> p+  -> Int+  -- ^ Batch size+  -> Stream (Of r) IO ()+selectStream conn stmt params batchSize = do+  rows <- liftIO (fetchAllCursor conn stmt params batchSize)+  S.each rows++-- | Stream query results using single-shot extended-protocol execution.+--+-- Does not require a transaction.+foldStream+  :: Connection+  -> Statement p r+  -> p+  -> Stream (Of r) IO ()+foldStream conn stmt params = do+  rows <- liftIO (fetchAll conn stmt params)+  S.each rows

+ test/Main.hs view

@@ -0,0 +1,59 @@+module Main where++import Data.Int (Int32)+import Data.Text (Text)+import Valiant+import Valiant.Stream (foldStream, selectStream)+import Streaming.Prelude qualified as S+import Test.Hspec+import TestSupport++main :: IO ()+main = hspec $ do+  describe "Valiant.Stream" $ do+    describe "selectStream (cursor-based)" $ do+      it "streams all rows from a cursor" $ do+        withTestConnection $ \conn -> withSchema conn $ do+          insertTestUsers conn+          withTransactionConn conn $ \tx -> do+            rows <- S.toList_ $+              selectStream (txConn tx) stmtListAll () 2+            length rows `shouldBe` 5++      it "returns empty for no rows" $ do+        withTestConnection $ \conn -> withSchema conn $ do+          withTransactionConn conn $ \tx -> do+            rows <- S.toList_ $+              selectStream (txConn tx) stmtListAll () 10+            rows `shouldBe` ([] :: [(Int32, Text)])++      it "works with small batch size" $ do+        withTestConnection $ \conn -> withSchema conn $ do+          insertTestUsers conn+          withTransactionConn conn $ \tx -> do+            rows <- S.toList_ $+              selectStream (txConn tx) stmtListAll () 1+            length rows `shouldBe` 5++    describe "foldStream (no transaction)" $ do+      it "streams all rows without a transaction" $ do+        withTestConnection $ \conn -> withSchema conn $ do+          insertTestUsers conn+          rows <- S.toList_ $+            foldStream conn stmtListAll ()+          length rows `shouldBe` 5++      it "returns empty for no rows" $ do+        withTestConnection $ \conn -> withSchema conn $ do+          rows <- S.toList_ $+            foldStream conn stmtListAll ()+          rows `shouldBe` ([] :: [(Int32, Text)])++      it "composes with streaming combinators" $ do+        withTestConnection $ \conn -> withSchema conn $ do+          insertTestUsers conn+          names <- S.toList_ $+            S.map snd $+            foldStream conn stmtListAll ()+          length names `shouldBe` 5+          take 1 names `shouldBe` ["Alice"]

+ test/TestSupport.hs view

@@ -0,0 +1,63 @@+module TestSupport+  ( withTestConnection+  , withSchema+  , insertTestUsers+  , stmtListAll+  , stmtSelectOne+  ) where++import Data.ByteString (ByteString)+import Data.ByteString.Char8 qualified as BS8+import Data.Int (Int32)+import Data.Text (Text)+import Valiant+import System.Environment (lookupEnv)++requireDatabaseUrl :: IO ByteString+requireDatabaseUrl = do+  mUrl <- lookupEnv "DATABASE_URL"+  case mUrl of+    Just url -> pure (BS8.pack url)+    Nothing -> error "DATABASE_URL is not set."++withTestConnection :: (Connection -> IO a) -> IO a+withTestConnection action = do+  url <- requireDatabaseUrl+  conn <- connectString url+  result <- action conn+  close conn+  pure result++withSchema :: Connection -> IO a -> IO a+withSchema conn action = do+  _ <- simpleQuery conn "DROP TABLE IF EXISTS users_streaming CASCADE"+  _ <- simpleQuery conn+    "CREATE TABLE users_streaming (\+    \  id SERIAL PRIMARY KEY,\+    \  name TEXT NOT NULL,\+    \  email TEXT\+    \)"+  result <- action+  _ <- simpleQuery conn "DROP TABLE IF EXISTS users_streaming CASCADE"+  pure result++insertTestUsers :: Connection -> IO ()+insertTestUsers conn = do+  _ <- simpleQuery conn+    "INSERT INTO users_streaming (name, email) VALUES \+    \('Alice', 'alice@example.com'),\+    \('Bob', 'bob@example.com'),\+    \('Carol', NULL),\+    \('Dave', 'dave@example.com'),\+    \('Eve', 'eve@example.com')"+  pure ()++stmtListAll :: Statement () (Int32, Text)+stmtListAll = mkStatement+  "SELECT id, name FROM users_streaming ORDER BY id"+  [] ["id", "name"] "<test>"++stmtSelectOne :: Statement Int32 (Int32, Text, Maybe Text)+stmtSelectOne = mkStatement+  "SELECT id, name, email FROM users_streaming WHERE id = $1"+  [23] ["id", "name", "email"] "<test>"

+ valiant-streaming.cabal view

@@ -0,0 +1,76 @@+cabal-version:   3.0+name:            valiant-streaming+version:         0.1.0.0+synopsis:        Streaming adapter for valiant (streaming library)+description:+  Stream PostgreSQL query results using the @streaming@ library.+  Provides cursor-based and fold-based producers as @Stream (Of r) IO ()@+  values.+license:         BSD-3-Clause+license-file:    LICENSE+author:          Josh Burgess+maintainer:      joshburgess.webdev@gmail.com+category:        Database+homepage:        https://github.com/joshburgess/valiant+bug-reports:     https://github.com/joshburgess/valiant/issues+build-type:      Simple+extra-doc-files:+  README.md+  CHANGELOG.md+tested-with:     GHC ==9.10.3++source-repository head+  type:     git+  location: https://github.com/joshburgess/valiant+  subdir:   adapters/valiant-streaming++flag werror+  description: Enable -Werror for development builds.+  default:     False+  manual:      True++common warnings+  ghc-options: -Wall -Wcompat -Wno-unticked-promoted-constructors -funbox-strict-fields -fspecialise-aggressively+  if flag(werror)+    ghc-options: -Werror++library+  import:           warnings+  hs-source-dirs:   src+  default-language: GHC2021+  default-extensions:+    DerivingStrategies+    OverloadedStrings+    StrictData++  exposed-modules:+    Valiant.Stream++  build-depends:+    , base            >=4.17 && <5+    , valiant         >=0.1  && <0.2+    , streaming       >=0.2  && <0.3++test-suite valiant-streaming-test+  import:           warnings+  type:             exitcode-stdio-1.0+  hs-source-dirs:   test+  main-is:          Main.hs+  default-language: GHC2021+  default-extensions:+    OverloadedStrings++  ghc-options: -rtsopts "-with-rtsopts=-K8K"++  other-modules:+    TestSupport++  build-depends:+    , base            >=4.17 && <5+    , bytestring      >=0.11 && <0.13+    , hspec           >=2.11 && <2.13+    , valiant+    , valiant-streaming+    , pg-wire+    , streaming       >=0.2  && <0.3+    , text            >=2.0  && <2.2