diff --git a/README.md b/README.md new file mode 100644 --- /dev/null +++ b/README.md @@ -0,0 +1,63 @@ +streaming +========= + +This library defines an optimized `FreeT` with an eye to use with +streaming libraries, namely: + + data Stream f m r + = Return r + | Step !(f (Stream f m r)) + | Delay (m (Stream f m r)) + +in place of the standard `FreeT` that we find in the `free` library, which +is approximately: + + newtype FreeT f m r = FreeT {runFreeT :: m (Either r (f (FreeT f m r)))} + +Rather than wrapping each step in a monadic 'layer', such a layer is +put alongside separate 'pure' constructors for a functor 'layer' +and a final return value. The maneuver is very friendly to the compiler, +but requires a bit of subtlety to protect a sound monad instance. Just such an optimization is adopted internally by the `pipes` library. +As in `pipes`, the constructors are here left in an `Internal` module; +the main `Streaming` module exporting the type itself and various +operations and instances. + +There is also a still-incomplete `Prelude` of functions, some +`FreeT` or `Stream` - general, some involving the functor +`((,) a)` here called `Of a`. (`Stream (Of a) m r` like +`FreeT ((,) a) m r` is equivalent to the `pipes` +`Producer a m r` type. Similarly, `Stream (Of a) m ()` and +`FreeT ((,) a) m ()` are possible implementations +of `ListT done right`. + +I ran a simple [benchmark](https://gist.github.com/michaelt/7f89dc8b366b30bb6acc) (adjusting a [script](https://github.com/jwiegley/streaming-tests) of John Weigly) using a very simple composition of functions: + + toList + . filter (\x -> x `mod` 2 == 0) + . map (+1) + . drop 1000 + . map (+1) + . filter even + . each + +The the results were fairly pleasing: + + benchmarking basic/streaming + time 84.50 ms (79.81 ms .. 87.90 ms) + + benchmarking basic/iostreams + time 266.2 ms (235.6 ms .. 292.0 ms) + + benchmarking basic/pipes + time 232.0 ms (206.6 ms .. 246.7 ms) + + benchmarking basic/conduit + time 102.3 ms (96.24 ms .. 110.0 ms) + +This sequence of pre-packaged combinators is, I think, very friendly to the +more recent conduit fusion framework. The framework of course doesn't apply to +user-defined operations, where we should expect times like those shown for pipes. +Since the combinators from `streaming` is defined with naive recursion, +more or less as the user might, we have reason to think the result is characteristic, +but much more benchmarking is needed before anything can be said with certainty. +The labor of constructor-hiding may turn up some further difficulty. diff --git a/streaming.cabal b/streaming.cabal --- a/streaming.cabal +++ b/streaming.cabal @@ -1,20 +1,20 @@ name: streaming -version: 0.1.0.18 +version: 0.1.0.19 cabal-version: >=1.10 build-type: Simple -synopsis: a free monad transformer optimized for streaming applications with an elementary streaming prelude +synopsis: an elementary streaming prelude and a free monad transformer optimized for streaming applications -description: * __The free stream on a streamable functor__ +description: /A. The freely-developed stream on a streamable functor/ . @Stream@ can be used wherever is used. The compiler's standard range of optimizations work better for operations - written in terms of `Stream`. @FreeT f m r@ / @Stream f m r@ - is of course extremely general, and many functor-general combinators + written in terms of `Stream`. @FreeT f m r@ and @Stream f m r@ + are of course extremely general, and many functor-general combinators are exported by the general module @Streaming@. . - * __The general idea of streaming__ + /B. The general idea of streaming/ . As soon as you consider the idea of an effectful stream of any kind whatsoever, for example, a stream of bytes from a handle, however @@ -31,7 +31,7 @@ /succession/ of /such unfoldings/ linked together end to end in accordance with the initial succession of seed values. . - Call those 5 sentences the ABCs. If you understood the ABCs + Call those 5 sentences the ABCs of streaming. If you understood these ABCs you have a total comprehension of @Stream f m r@. . * @Stream@ itself expresses what the word "succession" meant in the ABCs @@ -49,7 +49,7 @@ streaming-bytestring> library, or whatever stream-form you can express in a Haskell functor. . - * __A freely generated stream of /connected individual Haskell values/ is Producer, Generator or Source__ + /C. A freely generated stream of/ connected individual Haskell values /is a Producer, Generator or Source/ . But, of course, as soon as you grasp the general form of /succession/, you are already in possession of the most basic concrete form: a simple @@ -86,7 +86,7 @@ > machines: SourceT m a (= forall k. MachineT m k a) > streaming: Stream (Of a) m () . - * __@Streaming.Prelude@__ + /D./ @Streaming.Prelude@ . @Streaming.Prelude@ closely follows @Pipes.Prelude@. But since it restricts itself to use @@ -110,7 +110,7 @@ Somehow, we didn't even need a four-character operator for that, nor advice about best practices! - just ordinary Haskell common sense. . - * __Mother's @Prelude@ vs. @Streaming.Prelude@__ + /E. Mother's/ @Prelude@ /vs./ @Streaming.Prelude@ . The effort of @Streaming.Prelude@ is to leverage the intuition the user has acquired @@ -138,7 +138,7 @@ I will labor this point a bit more below, but you can also find it developed, with greater skill, in the documentation for the pipes libraries. . - * __How come there's not one of those fancy "ListT done right" implementations in here?__ + /F. How come there's not one of those fancy "ListT done right" implementations in here?/ . The use of the final return value appears to be a complication, but in fact it is essentially contained in the idea of effectful streaming. This is why @@ -184,7 +184,7 @@ each line to a different file, as it might be, but without accumulation - are documented within. So are are myriad other elementary operations of streaming io. . - * __Didn't I hear that free monads are a real efficiency dog? Isn't Oleg working on this important problem?__ + /G. Didn't I hear that free monads are a dog from the point of view of efficiency?/ . We noted above that if we instantiate @Stream f m r@ to @Stream ((,) a) m r@ or the like, we get the standard idea of a producer or generator. @@ -280,29 +280,9 @@ gives you an immense body of such structures and a simple discipline for working with them. Spinkle @id@ freely though your program if you get homesick. - . - Much of the discussion of the free monad concept is associated - with the "algebraic effects" program. A leading advertisement for this approach - is that we can toss generators into the soup without missing a beat. - See for example this - . - concept - . - > yield :: (Typeable a, Member (Yield a) r) => a -> Eff r () - . - With it I can over course write, e.g. - . - > each :: (Traversable t, Typeable a, Member (Yield a) r) => t a -> Eff r () - > each = mapM_ yield - . - Once I have one of these "coroutine effects" on my hands, - the fact that I am writing Haskell, not e.g. Python, will leave me with - little trouble splitting it at the 20th element, and reserving the rest for later use. - I invite you, though, to divide such a "coroutine effect" on its lines or - into chunks of 500. There must be /some/ sense in which these effects are "extensible". - But it seems not as far as the ABCs. + . - * __Interoperation with the streaming-io libraries__ + /H. Interoperation with the streaming-io libraries/ . The simplest form of interoperation with @@ -334,7 +314,7 @@ > Free.iterTM Stream.wrap :: FreeT f m a -> Stream f m a > Stream.iterTM Free.wrap :: Stream f m a -> FreeT f m a . - * __Examples__ + /I. Where can I find examples of use?/ . For some simple ghci examples, see the commentary throughout the Prelude module. For slightly more advanced usage see the commentary in the haddocks of @@ -344,7 +324,7 @@ Here's a simple with intrinsically streaming byte streams. . - * __Problems__ + /J. Problems/ . Questions about this library can be put as issues through the github site or on the . @@ -359,6 +339,8 @@ homepage: https://github.com/michaelt/streaming bug-reports: https://github.com/michaelt/streaming/issues category: Data, Pipes, Streaming +extra-source-files: README.md + source-repository head type: git location: https://github.com/michaelt/streaming