valiant-pipes (empty) → 0.1.0.0

raw patch · 7 files changed

+334/−0 lines, 7 filesdep +basedep +bytestringdep +hspec

Dependencies added: base, bytestring, hspec, pg-wire, pipes, text, valiant, valiant-pipes

Files

CHANGELOG.md +14/−0
LICENSE +27/−0
README.md +39/−0
src/Valiant/Pipes.hs +57/−0
test/Main.hs +58/−0
test/TestSupport.hs +63/−0
valiant-pipes.cabal +76/−0

+ CHANGELOG.md view

@@ -0,0 +1,14 @@+# Changelog++The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).++## [0.1.0.0] - 2026-04-25++Initial release. Pipes streaming adapter for+[`valiant`](https://hackage.haskell.org/package/valiant). Exposes+`selectPipe` (cursor-based, requires a transaction) and+`foldPipe` (single-shot extended-protocol) as+`Producer r IO ()` values.++[0.1.0.0]: https://github.com/joshburgess/valiant/releases/tag/v0.1.0.0

+ LICENSE view

@@ -0,0 +1,27 @@+Copyright (c) 2026 Josh Burgess++Redistribution and use in source and binary forms, with or without+modification, are permitted provided that the following conditions are met:++1. Redistributions of source code must retain the above copyright notice,+   this list of conditions and the following disclaimer.++2. Redistributions in binary form must reproduce the above copyright notice,+   this list of conditions and the following disclaimer in the documentation+   and/or other materials provided with the distribution.++3. Neither the name of the copyright holder nor the names of its+   contributors may be used to endorse or promote products derived from this+   software without specific prior written permission.++THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE+ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE+LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF+SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS+INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN+CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)+ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE+POSSIBILITY OF SUCH DAMAGE.

+ README.md view

@@ -0,0 +1,39 @@+# valiant-pipes++[Pipes](https://hackage.haskell.org/package/pipes) streaming adapter+for [`valiant`](https://hackage.haskell.org/package/valiant).++Produces `Producer r IO ()` values from query results, so they compose+with the pipes ecosystem.++## Quick start++```haskell+import Valiant+import Valiant.Pipes+import Pipes+import Pipes.Prelude qualified as P++printAllUserNames :: Pool -> IO ()+printAllUserNames pool =+  withTransaction pool $ \tx ->+    runEffect $+      selectPipe (txConn tx) listAllUsers () 500+      >-> P.map userName+      >-> P.stdoutLn+```++## Two streaming strategies++- `selectPipe conn stmt params batchSize` — cursor-based. Must run+  inside a transaction.+- `foldPipe conn stmt params` — single-shot extended-protocol query.+  No transaction required.++Both functions accumulate one result set at a time before yielding,+so memory usage scales with the active batch (cursor) or the full+result set (`foldPipe`). For truly incremental memory use, drive+`Valiant.withCursor` / `Valiant.fetchBatch` directly.++See the [valiant tutorial](https://github.com/joshburgess/valiant/blob/main/docs/TUTORIAL.md)+for `Statement` definitions and the `valiant prepare` workflow.

+ src/Valiant/Pipes.hs view

@@ -0,0 +1,57 @@+{-# OPTIONS_GHC -fno-full-laziness #-}++-- | Pipes streaming adapter for valiant.+--+-- Produces @Producer r IO ()@ values from query results, enabling+-- composable stream processing with the pipes ecosystem.+--+-- @+-- import Valiant+-- import Valiant.Pipes+-- import Pipes+-- import Pipes.Prelude qualified as P+--+-- withTransaction pool $ \\tx ->+--   runEffect $+--     selectPipe (txConn tx) listAllUsers () 500+--     >-> P.map userName+--     >-> P.stdoutLn+-- @+module Valiant.Pipes+  ( -- * Cursor-based streaming (requires transaction)+    selectPipe+    -- * Fold-based streaming (no transaction required)+  , foldPipe+  ) where++import Pipes (Producer, liftIO, yield)+import Valiant (Connection, Statement, fetchAll, fetchAllCursor)++-- | Stream query results using a server-side cursor.+--+-- Must be called inside a transaction. Fetches rows in batches of the+-- given size, yielding decoded rows one at a time.+--+-- Note: rows are accumulated in memory before yielding (see+-- 'Valiant.Conduit.selectSource' for the same caveat).+selectPipe+  :: Connection+  -> Statement p r+  -> p+  -> Int+  -> Producer r IO ()+selectPipe conn stmt params batchSize = do+  rows <- liftIO (fetchAllCursor conn stmt params batchSize)+  mapM_ yield rows++-- | Stream query results using single-shot extended-protocol execution.+--+-- Does not require a transaction.+foldPipe+  :: Connection+  -> Statement p r+  -> p+  -> Producer r IO ()+foldPipe conn stmt params = do+  rows <- liftIO (fetchAll conn stmt params)+  mapM_ yield rows

+ test/Main.hs view

@@ -0,0 +1,58 @@+module Main where++import Data.Int (Int32)+import Data.Text (Text)+import Valiant+import Valiant.Pipes (foldPipe, selectPipe)+import Pipes ((>->))+import Pipes.Prelude qualified as P+import Test.Hspec+import TestSupport++main :: IO ()+main = hspec $ do+  describe "Valiant.Pipes" $ do+    describe "selectPipe (cursor-based)" $ do+      it "streams all rows from a cursor" $ do+        withTestConnection $ \conn -> withSchema conn $ do+          insertTestUsers conn+          withTransactionConn conn $ \tx -> do+            rows <- P.toListM $+              selectPipe (txConn tx) stmtListAll () 2+            length rows `shouldBe` 5++      it "returns empty for no rows" $ do+        withTestConnection $ \conn -> withSchema conn $ do+          withTransactionConn conn $ \tx -> do+            rows <- P.toListM $+              selectPipe (txConn tx) stmtListAll () 10+            rows `shouldBe` ([] :: [(Int32, Text)])++      it "works with small batch size" $ do+        withTestConnection $ \conn -> withSchema conn $ do+          insertTestUsers conn+          withTransactionConn conn $ \tx -> do+            rows <- P.toListM $+              selectPipe (txConn tx) stmtListAll () 1+            length rows `shouldBe` 5++    describe "foldPipe (no transaction)" $ do+      it "streams all rows without a transaction" $ do+        withTestConnection $ \conn -> withSchema conn $ do+          insertTestUsers conn+          rows <- P.toListM $ foldPipe conn stmtListAll ()+          length rows `shouldBe` 5++      it "returns empty for no rows" $ do+        withTestConnection $ \conn -> withSchema conn $ do+          rows <- P.toListM $ foldPipe conn stmtListAll ()+          rows `shouldBe` ([] :: [(Int32, Text)])++      it "composes with pipes combinators" $ do+        withTestConnection $ \conn -> withSchema conn $ do+          insertTestUsers conn+          names <- P.toListM $+            foldPipe conn stmtListAll ()+            >-> P.map snd+          length names `shouldBe` 5+          take 1 names `shouldBe` ["Alice"]

+ test/TestSupport.hs view

@@ -0,0 +1,63 @@+module TestSupport+  ( withTestConnection+  , withSchema+  , insertTestUsers+  , stmtListAll+  , stmtSelectOne+  ) where++import Data.ByteString (ByteString)+import Data.ByteString.Char8 qualified as BS8+import Data.Int (Int32)+import Data.Text (Text)+import Valiant+import System.Environment (lookupEnv)++requireDatabaseUrl :: IO ByteString+requireDatabaseUrl = do+  mUrl <- lookupEnv "DATABASE_URL"+  case mUrl of+    Just url -> pure (BS8.pack url)+    Nothing -> error "DATABASE_URL is not set."++withTestConnection :: (Connection -> IO a) -> IO a+withTestConnection action = do+  url <- requireDatabaseUrl+  conn <- connectString url+  result <- action conn+  close conn+  pure result++withSchema :: Connection -> IO a -> IO a+withSchema conn action = do+  _ <- simpleQuery conn "DROP TABLE IF EXISTS users_pipes CASCADE"+  _ <- simpleQuery conn+    "CREATE TABLE users_pipes (\+    \  id SERIAL PRIMARY KEY,\+    \  name TEXT NOT NULL,\+    \  email TEXT\+    \)"+  result <- action+  _ <- simpleQuery conn "DROP TABLE IF EXISTS users_pipes CASCADE"+  pure result++insertTestUsers :: Connection -> IO ()+insertTestUsers conn = do+  _ <- simpleQuery conn+    "INSERT INTO users_pipes (name, email) VALUES \+    \('Alice', 'alice@example.com'),\+    \('Bob', 'bob@example.com'),\+    \('Carol', NULL),\+    \('Dave', 'dave@example.com'),\+    \('Eve', 'eve@example.com')"+  pure ()++stmtListAll :: Statement () (Int32, Text)+stmtListAll = mkStatement+  "SELECT id, name FROM users_pipes ORDER BY id"+  [] ["id", "name"] "<test>"++stmtSelectOne :: Statement Int32 (Int32, Text, Maybe Text)+stmtSelectOne = mkStatement+  "SELECT id, name, email FROM users_pipes WHERE id = $1"+  [23] ["id", "name", "email"] "<test>"

+ valiant-pipes.cabal view

@@ -0,0 +1,76 @@+cabal-version:   3.0+name:            valiant-pipes+version:         0.1.0.0+synopsis:        Pipes streaming adapter for valiant+description:+  Stream PostgreSQL query results using the @pipes@ library.+  Provides cursor-based and fold-based producers as+  @Producer r IO ()@ values.+license:         BSD-3-Clause+license-file:    LICENSE+author:          Josh Burgess+maintainer:      joshburgess.webdev@gmail.com+category:        Database+homepage:        https://github.com/joshburgess/valiant+bug-reports:     https://github.com/joshburgess/valiant/issues+build-type:      Simple+extra-doc-files:+  README.md+  CHANGELOG.md+tested-with:     GHC ==9.10.3++source-repository head+  type:     git+  location: https://github.com/joshburgess/valiant+  subdir:   adapters/valiant-pipes++flag werror+  description: Enable -Werror for development builds.+  default:     False+  manual:      True++common warnings+  ghc-options: -Wall -Wcompat -Wno-unticked-promoted-constructors -funbox-strict-fields -fspecialise-aggressively+  if flag(werror)+    ghc-options: -Werror++library+  import:           warnings+  hs-source-dirs:   src+  default-language: GHC2021+  default-extensions:+    DerivingStrategies+    OverloadedStrings+    StrictData++  exposed-modules:+    Valiant.Pipes++  build-depends:+    , base            >=4.17 && <5+    , valiant         >=0.1  && <0.2+    , pipes           >=4.3  && <4.4++test-suite valiant-pipes-test+  import:           warnings+  type:             exitcode-stdio-1.0+  hs-source-dirs:   test+  main-is:          Main.hs+  default-language: GHC2021+  default-extensions:+    OverloadedStrings++  ghc-options: -rtsopts "-with-rtsopts=-K8K"++  other-modules:+    TestSupport++  build-depends:+    , base            >=4.17 && <5+    , bytestring      >=0.11 && <0.13+    , hspec           >=2.11 && <2.13+    , valiant+    , valiant-pipes+    , pg-wire+    , pipes           >=4.3  && <4.4+    , text            >=2.0  && <2.2