streaming-0.1.4.0: streaming.cabal
name: streaming
version: 0.1.4.0
cabal-version: >=1.10
build-type: Simple
synopsis: an elementary streaming prelude and general stream type.
description: @Streaming.Prelude@ exports an elementary streaming prelude focussed on
a simple \"source\" or \"producer\" type, namely @Stream (Of a) m r@.
This is a sort of effectful version of @([a],r)@ in which monadic action
is interleaved between successive elements.
The main module, @Streaming@, exports a much more general type,
@Stream f m r@, which can be used to stream successive distinct
steps characterized by /any/
functor @f@, though we are here interested only in a limited range of
cases.
.
The streaming-IO libraries have various devices for dealing
with effectful variants of @[a]@ or @([a],r)@. But it is only with
the general type @Stream f m r@, or some equivalent,
that one can envisage (for example) the connected streaming of their
sorts of stream -- as one makes lists of lists in the Haskell
@Prelude@ and @Data.List@. One needs some such type if we are
to express properly streaming equivalents of e.g.
.
> group :: Ord a => [a] -> [[a]]
> chunksOf :: Int -> [a] -> [[a]]
> lines :: [Char] -> [[Char]] -- but similarly with bytestring, etc.
.
to mention a few obviously desirable operations. But once one grasps
the iterable stream concept needed to express those functions - to wit,
@Stream f m r@ or some equivalent - then one will also see that,
with it, one is already in possession of a complete
elementary streaming library - since one possesses @Stream ((,) a) m r@
or equivalently @Stream (Of a) m r@. This
is the type of a \'generator\' or \'producer\' or whatever
you call an effectful stream of items.
The present @Streaming.Prelude@ is thus the simplest streaming
library that can replicate anything like the API of the
@Prelude@ and @Data.List@.
.
The emphasis of the library is on interoperation; for
the rest its advantages are: extreme simplicity and re-use of
intuitions the user has gathered from mastery of @Prelude@ and
@Data.List@. The two conceptual pre-requisites are some
comprehension of monad transformers and some familiarity
with \'rank 2 types\'.
.
See the
<https://hackage.haskell.org/package/streaming#readme readme> below
for an explanation, including the examples linked there. Elementary usage can be divined from the ghci examples in
@Streaming.Prelude@ and perhaps from this rough beginning of a
<https://github.com/michaelt/streaming-tutorial/blob/master/tutorial.md tutorial>.
Note also the
<https://hackage.haskell.org/package/streaming-bytestring streaming bytestring>
and
<https://hackage.haskell.org/package/streaming-utils streaming utils>
packages.
.
The simplest form of interoperation with <http://hackage.haskell.org/package/pipes pipes>
is accomplished with this isomorphism:
.
> Pipes.unfoldr Streaming.next :: Stream (Of a) m r -> Producer a m r
> Streaming.unfoldr Pipes.next :: Producer a m r -> Stream (Of a) m r
.
Interoperation with <http://hackage.haskell.org/package/io-streams io-streams> is thus:
.
> Streaming.reread IOStreams.read :: InputStream a -> Stream (Of a) IO ()
> IOStreams.unfoldM Streaming.uncons :: Stream (Of a) IO () -> IO (InputStream a)
.
A simple exit to <http://hackage.haskell.org/package/conduit conduit> would be, e.g.:
.
> Conduit.unfoldM Streaming.uncons :: Stream (Of a) m () -> Source m a
.
These conversions should never be more expensive than a single @>->@ or @=$=@.
Here is a simple example that runs a single underlying stream with several
streaming-io libraries at once, superimposing their effects
without any accumulation:
.
> module Main (main) where
> import Streaming
> import Pipes
> import Data.Conduit
> import qualified Streaming.Prelude as S
> import qualified Data.Conduit.List as CL
> import qualified Pipes.Prelude as P
> import qualified System.IO.Streams as IOS
> import Data.ByteString.Char8 (pack)
> import Data.Function ((&))
>
> mkConduit = CL.unfoldM S.uncons
> mkPipe = P.unfoldr S.next
> mkIOStream = IOS.unfoldM S.uncons
>
> main = iostreamed where
> urstream = S.take 4 S.readLn :: Stream (Of Int) IO ()
> streamed = S.copy urstream
> & S.map (\n -> "streaming says: " ++ show n)
> & S.stdoutLn
> piped = runEffect $
> mkPipe (S.copy streamed)
> >-> P.map (\n -> "pipes says: " ++ show n)
> >-> P.stdoutLn
> conduited = mkConduit (S.copy piped)
> $$ CL.map (\n -> "conduit says: " ++ show n)
> =$ CL.mapM_ (liftIO . putStrLn)
> iostreamed = do
> str0 <- mkIOStream conduited
> str1 <- IOS.map (\n -> pack $ "io-streams says: " ++ show n ++ "\n") str0
> IOS.supply str1 IOS.stdout
.
This program successively parses four @Int@s from standard input,
and /simulaneously/ passes them to (here trivial) stream-consuming
processes from four different libraries, using the @copy@ function from
@Streaming.Prelude@. I mark my own input with @/<Enter/>@:
.
> >>> main
> 1 <Enter>
> streaming says: 1
> pipes says: 1
> conduit says: 1
> io-streams says: 1
> 2 <Enter>
> streaming says: 2
> pipes says: 2
> conduit says: 2
> io-streams says: 2
> 3 <Enter>
> streaming says: 3
> pipes says: 3
> conduit says: 3
> io-streams says: 3
> 4 <Enter>
> streaming says: 4
> pipes says: 4
> conduit says: 4
> io-streams says: 4
.
Of course, I could as well have passed the stream to several
independent conduits, for example. Further
points of comparison with the going streaming-IO libraries
are discussed in the
<https://hackage.haskell.org/package/streaming#readme readme>
below.
.
Here are the results of some
<https://gist.github.com/michaelt/f19bef01423b17f29ffd microbenchmarks>
based on the
<https://github.com/ekmett/machines/blob/master/benchmarks/Benchmarks.hs benchmarks>
included in the machines package:
.
<<http://i.imgur.com/sSG5MvH.png>>
.
Because these are microbenchmarks for individual functions,
they represent a sort of "worst case"; many other factors can influence
the speed of a complex program.
.
license: BSD3
license-file: LICENSE
author: michaelt
maintainer: what_is_it_to_do_anything@yahoo.com
stability: Experimental
homepage: https://github.com/michaelt/streaming
bug-reports: https://github.com/michaelt/streaming/issues
category: Data, Pipes, Streaming
extra-source-files: README.md
source-repository head
type: git
location: https://github.com/michaelt/streaming
library
exposed-modules: Streaming,
Streaming.Prelude,
Streaming.Internal
-- other-modules:
other-extensions: RankNTypes, CPP,
StandaloneDeriving, FlexibleContexts,
DeriveDataTypeable, DeriveFoldable,
DeriveFunctor, DeriveTraversable,
UndecidableInstances
build-depends: base >=4.6 && <5
, mtl >=2.1 && <2.3
, mmorph >=1.0 && <1.2
, transformers >=0.4 && <0.5
, transformers-base
, bytestring
, time
, resourcet
, exceptions
, containers
default-language: Haskell2010