packages feed

superbuffer (empty) → 0.1.0.0

raw patch · 10 files changed

+674/−0 lines, 10 filesdep +HTFdep +QuickCheckdep +basesetup-changed

Dependencies added: HTF, QuickCheck, base, bytestring, criterion, superbuffer

Files

+ LICENSE view
@@ -0,0 +1,30 @@+Copyright Alexander Thiemann (c) 2016++All rights reserved.++Redistribution and use in source and binary forms, with or without+modification, are permitted provided that the following conditions are met:++    * Redistributions of source code must retain the above copyright+      notice, this list of conditions and the following disclaimer.++    * Redistributions in binary form must reproduce the above+      copyright notice, this list of conditions and the following+      disclaimer in the documentation and/or other materials provided+      with the distribution.++    * Neither the name of Alexander Thiemann nor the names of other+      contributors may be used to endorse or promote products derived+      from this software without specific prior written permission.++THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ README.md view
@@ -0,0 +1,202 @@+# Haskell SuperBuffer++[![CircleCI](https://circleci.com/gh/agrafix/superbuffer.svg?style=svg)](https://circleci.com/gh/agrafix/superbuffer)++The `superbuffer` packages was designed to efficiently build up bytestrings from `IO` actions producing+smaller chunks. The goal was to reduce memory overhead as much as possible while still being as fast as possible.+In our use case, it reduced total memory usage of the program from `350 MB` (`bytestring` builder) to `50 MB` (`superbuffer`).+For speed see benchmarks below. Note that the speed heavily depends on a good choice of the initial buffer size and+the size of the chunks written. For small chunks the `superbuffer` outperforms the `bytestring` alternatives consistently. Note+that the library is currently not thread-safe, but it seems that it could be added (from outside) easily by making sure only one thread+at a time can call `appendBuffer` (making the call slower).++## Usage++```haskell+{-# LANGUAGE OverloadedStrings #-}+module Example where++import Data.ByteString.SuperBuffer+import qualified Data.ByteString as BS++myBS :: IO BS.ByteString+myBS =+    -- note: performance of superbuffer heavily depends on a+    -- smart choice of the initial buffer size. Benchmark to+    -- find what suits your needs.+    withBuffer 1024 $ \buf ->+    do appendBuffer buf "Hello "+       appendBuffer buf "World!"+```++## Benchmarks++```+benchmarking main/small/superbuffer (init=128 bytes)+time                 22.20 ms   (20.81 ms .. 24.69 ms)+                     0.973 R²   (0.946 R² .. 0.992 R²)+mean                 22.47 ms   (21.70 ms .. 23.45 ms)+std dev              1.905 ms   (1.515 ms .. 2.685 ms)+variance introduced by outliers: 38% (moderately inflated)++benchmarking main/small/superbuffer (init=4000 bytes)+time                 19.92 ms   (19.28 ms .. 20.58 ms)+                     0.996 R²   (0.993 R² .. 0.998 R²)+mean                 22.30 ms   (21.56 ms .. 23.60 ms)+std dev              2.230 ms   (1.324 ms .. 3.614 ms)+variance introduced by outliers: 46% (moderately inflated)++benchmarking main/small/superbuffer (init=8000 bytes)+time                 22.72 ms   (21.85 ms .. 23.92 ms)+                     0.994 R²   (0.989 R² .. 0.997 R²)+mean                 23.25 ms   (22.73 ms .. 23.95 ms)+std dev              1.496 ms   (1.007 ms .. 2.375 ms)+variance introduced by outliers: 24% (moderately inflated)++benchmarking main/small/superbuffer (init=16000 bytes)+time                 23.28 ms   (21.77 ms .. 25.14 ms)+                     0.978 R²   (0.953 R² .. 0.994 R²)+mean                 23.41 ms   (22.79 ms .. 24.34 ms)+std dev              1.727 ms   (1.209 ms .. 2.455 ms)+variance introduced by outliers: 29% (moderately inflated)++benchmarking main/small/superbuffer (init=20000000 bytes)+time                 35.78 ms   (34.57 ms .. 37.21 ms)+                     0.989 R²   (0.972 R² .. 0.998 R²)+mean                 36.45 ms   (35.25 ms .. 37.34 ms)+std dev              2.167 ms   (1.281 ms .. 3.677 ms)+variance introduced by outliers: 18% (moderately inflated)++benchmarking main/small/bytestring builder+time                 31.34 ms   (28.98 ms .. 33.54 ms)+                     0.987 R²   (0.974 R² .. 0.999 R²)+mean                 30.73 ms   (29.54 ms .. 31.59 ms)+std dev              2.205 ms   (1.699 ms .. 2.841 ms)+variance introduced by outliers: 28% (moderately inflated)++benchmarking main/small/bytestring fromChunks+time                 26.35 ms   (25.53 ms .. 27.40 ms)+                     0.996 R²   (0.991 R² .. 0.999 R²)+mean                 26.35 ms   (25.88 ms .. 27.79 ms)+std dev              1.685 ms   (675.3 μs .. 3.068 ms)+variance introduced by outliers: 25% (moderately inflated)++benchmarking main/small/bytestring concat+time                 30.54 ms   (29.83 ms .. 31.13 ms)+                     0.998 R²   (0.997 R² .. 0.999 R²)+mean                 31.45 ms   (30.94 ms .. 33.55 ms)+std dev              1.658 ms   (563.0 μs .. 3.317 ms)+variance introduced by outliers: 17% (moderately inflated)++benchmarking main/med/superbuffer (init=128 bytes)+time                 20.98 ms   (20.11 ms .. 21.97 ms)+                     0.992 R²   (0.983 R² .. 0.997 R²)+mean                 23.11 ms   (22.53 ms .. 24.12 ms)+std dev              1.686 ms   (1.020 ms .. 2.914 ms)+variance introduced by outliers: 29% (moderately inflated)++benchmarking main/med/superbuffer (init=40000 bytes)+time                 21.62 ms   (20.92 ms .. 22.43 ms)+                     0.996 R²   (0.991 R² .. 0.998 R²)+mean                 23.65 ms   (23.06 ms .. 24.72 ms)+std dev              1.735 ms   (1.044 ms .. 2.732 ms)+variance introduced by outliers: 29% (moderately inflated)++benchmarking main/med/superbuffer (init=80000 bytes)+time                 20.65 ms   (19.91 ms .. 21.35 ms)+                     0.994 R²   (0.985 R² .. 0.998 R²)+mean                 22.64 ms   (22.09 ms .. 23.84 ms)+std dev              1.767 ms   (971.0 μs .. 3.008 ms)+variance introduced by outliers: 33% (moderately inflated)++benchmarking main/med/superbuffer (init=160000 bytes)+time                 23.35 ms   (22.15 ms .. 24.56 ms)+                     0.992 R²   (0.987 R² .. 0.998 R²)+mean                 22.50 ms   (22.15 ms .. 23.08 ms)+std dev              1.038 ms   (639.2 μs .. 1.502 ms)+variance introduced by outliers: 14% (moderately inflated)++benchmarking main/med/superbuffer (init=20000000 bytes)+time                 35.02 ms   (31.20 ms .. 37.83 ms)+                     0.983 R²   (0.970 R² .. 0.998 R²)+mean                 32.02 ms   (31.12 ms .. 33.32 ms)+std dev              2.159 ms   (1.456 ms .. 3.304 ms)+variance introduced by outliers: 24% (moderately inflated)++benchmarking main/med/bytestring builder+time                 22.03 ms   (21.53 ms .. 22.48 ms)+                     0.998 R²   (0.997 R² .. 0.999 R²)+mean                 22.50 ms   (22.06 ms .. 23.71 ms)+std dev              1.614 ms   (467.3 μs .. 3.159 ms)+variance introduced by outliers: 29% (moderately inflated)++benchmarking main/med/bytestring fromChunks+time                 23.88 ms   (21.62 ms .. 26.14 ms)+                     0.975 R²   (0.954 R² .. 0.998 R²)+mean                 22.81 ms   (22.25 ms .. 23.95 ms)+std dev              1.812 ms   (784.9 μs .. 2.692 ms)+variance introduced by outliers: 33% (moderately inflated)++benchmarking main/med/bytestring concat+time                 21.79 ms   (21.28 ms .. 22.23 ms)+                     0.998 R²   (0.997 R² .. 0.999 R²)+mean                 22.52 ms   (22.05 ms .. 23.70 ms)+std dev              1.646 ms   (554.1 μs .. 3.042 ms)+variance introduced by outliers: 29% (moderately inflated)++benchmarking main/large/superbuffer (init=128 bytes)+time                 25.97 ms   (25.22 ms .. 26.62 ms)+                     0.994 R²   (0.984 R² .. 0.999 R²)+mean                 26.01 ms   (25.18 ms .. 26.90 ms)+std dev              1.943 ms   (1.274 ms .. 3.107 ms)+variance introduced by outliers: 30% (moderately inflated)++benchmarking main/large/superbuffer (init=400000 bytes)+time                 21.51 ms   (21.13 ms .. 21.88 ms)+                     0.999 R²   (0.997 R² .. 1.000 R²)+mean                 22.14 ms   (21.84 ms .. 22.75 ms)+std dev              939.7 μs   (407.0 μs .. 1.669 ms)+variance introduced by outliers: 14% (moderately inflated)++benchmarking main/large/superbuffer (init=800000 bytes)+time                 21.76 ms   (21.35 ms .. 22.34 ms)+                     0.997 R²   (0.995 R² .. 0.999 R²)+mean                 22.09 ms   (21.77 ms .. 22.54 ms)+std dev              858.0 μs   (532.7 μs .. 1.421 ms)+variance introduced by outliers: 14% (moderately inflated)++benchmarking main/large/superbuffer (init=1600000 bytes)+time                 31.34 ms   (30.55 ms .. 32.36 ms)+                     0.997 R²   (0.992 R² .. 0.999 R²)+mean                 30.42 ms   (29.34 ms .. 31.35 ms)+std dev              2.043 ms   (1.383 ms .. 3.018 ms)+variance introduced by outliers: 22% (moderately inflated)++benchmarking main/large/superbuffer (init=20000000 bytes)+time                 38.79 ms   (36.27 ms .. 41.72 ms)+                     0.983 R²   (0.964 R² .. 0.995 R²)+mean                 33.84 ms   (32.32 ms .. 35.51 ms)+std dev              3.268 ms   (2.816 ms .. 3.755 ms)+variance introduced by outliers: 36% (moderately inflated)++benchmarking main/large/bytestring builder+time                 21.89 ms   (21.35 ms .. 22.37 ms)+                     0.998 R²   (0.995 R² .. 0.999 R²)+mean                 23.02 ms   (22.46 ms .. 24.92 ms)+std dev              2.169 ms   (461.3 μs .. 4.197 ms)+variance introduced by outliers: 43% (moderately inflated)++benchmarking main/large/bytestring fromChunks+time                 21.91 ms   (21.49 ms .. 22.31 ms)+                     0.999 R²   (0.998 R² .. 0.999 R²)+mean                 22.47 ms   (22.13 ms .. 23.63 ms)+std dev              1.230 ms   (383.5 μs .. 2.326 ms)+variance introduced by outliers: 19% (moderately inflated)++benchmarking main/large/bytestring concat+time                 22.12 ms   (21.82 ms .. 22.42 ms)+                     0.999 R²   (0.998 R² .. 1.000 R²)+mean                 22.60 ms   (22.27 ms .. 23.51 ms)+std dev              1.198 ms   (404.6 μs .. 2.244 ms)+variance introduced by outliers: 19% (moderately inflated)+```
+ Setup.hs view
@@ -0,0 +1,2 @@+import Distribution.Simple+main = defaultMain
+ bench/Bench.hs view
@@ -0,0 +1,70 @@+module Main where++import Control.Monad+import Data.ByteString.SuperBuffer+import Data.Int+import qualified Data.ByteString as BS+import qualified Data.ByteString.Lazy as BSL+import qualified Data.ByteString.Builder as BSB++import Criterion+import Criterion.Main++main :: IO ()+main =+    defaultMain+    [ bgroup "main"+      [ mkGroup "small" 5000 4000+      , mkGroup "med" 500 40000+      , mkGroup "large" 50 400000+      ]+    ]++mkGroup :: String -> Int -> Int -> Benchmark+mkGroup name steps chunkSize =+    bgroup name+    [ bench (bufName iBufSize128) $ nfIO $ BS.reverse <$> buildBuf iBufSize128 steps chunkSize+    , bench (bufName iBufSize) $ nfIO $ BS.reverse <$> buildBuf iBufSize steps chunkSize+    , bench (bufName iBufSize2) $ nfIO $ BS.reverse <$> buildBuf iBufSize2 steps chunkSize+    , bench (bufName iBufSize4) $ nfIO $ BS.reverse <$> buildBuf iBufSize4 steps chunkSize+    , bench (bufName iBufSizeAll) $ nfIO $ BS.reverse <$> buildBuf iBufSizeAll steps chunkSize+    , bench "bytestring builder" $ nfIO $ BS.reverse <$> buildBufBuilder steps chunkSize+    , bench "bytestring fromChunks" $ nfIO $ BS.reverse <$> buildBufChunks steps chunkSize+    , bench "bytestring concat" $ nfIO $ BS.reverse <$> buildBufConcat steps chunkSize+    ]+    where+      bufName is = "superbuffer (init=" ++ show is ++ " bytes)"+      iBufSize128 = 128+      iBufSize = fromIntegral chunkSize+      iBufSize2 = 2 * fromIntegral chunkSize+      iBufSize4 = 4 * fromIntegral chunkSize+      iBufSizeAll = fromIntegral $ steps * chunkSize++mkChunk :: Int -> Int -> BS.ByteString+mkChunk step chunkSize =+    BS.replicate chunkSize (fromIntegral $ (step `mod` 100) + 50)+{-# INLINE mkChunk #-}++buildBuf :: Int64 -> Int -> Int -> IO BS.ByteString+buildBuf bufSize steps chunkSize =+    withBuffer bufSize $ \buf ->+    forM_ [0..steps] $ \step ->+    appendBuffer buf (mkChunk step chunkSize)++buildBufBuilder :: Int -> Int -> IO BS.ByteString+buildBufBuilder steps chunkSize =+    BSL.toStrict . BSB.toLazyByteString <$>+    foldM (\b a ->  pure $ b `mappend` BSB.byteString (mkChunk a chunkSize)) mempty [0..steps]++buildBufChunks :: Int -> Int -> IO BS.ByteString+buildBufChunks steps chunkSize =+    BSL.toStrict . BSL.fromChunks <$> (+    forM [0..steps] $ \step ->+            pure (mkChunk step chunkSize))+++buildBufConcat :: Int -> Int -> IO BS.ByteString+buildBufConcat steps chunkSize =+    BS.concat <$> (+    forM [0..steps] $ \step ->+            pure (mkChunk step chunkSize))
+ cbits/superbuffer.c view
@@ -0,0 +1,54 @@+#include <stdio.h>+#include <stdlib.h>+#include <string.h>++struct sbuf {+  char *contents;+  size_t currentSize;+  size_t maxSize;+};++struct sbuf *new_sbuf(const size_t initSize)+{+  char *contents = (char *)malloc(initSize + 1);+  contents[0] = '\0';++  struct sbuf *buf = (struct sbuf *)malloc(sizeof(struct sbuf));+  buf->contents = contents;+  buf->currentSize = 0;+  buf->maxSize = initSize;++  return buf;+}++void append_sbuf(struct sbuf *buf, const char *value, const size_t len)+{+  if (len == 0) {+    return;+  }+  const size_t nextSize = buf->currentSize + len;+  while (nextSize > buf->maxSize) {+    // reallocate more memory+    buf->maxSize = buf->maxSize + (buf->maxSize >> 1);+    buf->contents = (char *)realloc(buf->contents, buf->maxSize);+  }+  char *targetLocation = buf->contents + buf->currentSize;+  memcpy(targetLocation, value, len);+  buf->currentSize = nextSize;+}++char *read_sbuf(struct sbuf *buf, size_t *len)+{+  *len = buf->currentSize;+  return buf->contents;+}++void destroyContents_sbuf(const struct sbuf *buf)+{+  free(buf->contents);+}++void destroy_sbuf(struct sbuf *buf)+{+  free(buf);+}
+ package.yaml view
@@ -0,0 +1,46 @@+name:                superbuffer+version:             0.1.0.0+synopsis:            Efficiently build a bytestring from smaller chunks+description:         Efficiently (both fast and memory efficient) build a bytestring from smaller chunks+homepage:            https://github.com/agrafix/superbuffer#readme+license:             BSD3+author:              Alexander Thiemann+maintainer:          mail@athiemann.net+copyright:           2016 Alexander Thiemann <mail@athiemann.net>+category:            Web+extra-source-files:+  - README.md+  - stack.yaml+  - package.yaml++dependencies:+  - base >= 4.7 && < 5+  - bytestring < 0.11++ghc-options: -Wall++library:+  source-dirs: src+  exposed-modules:+    Data.ByteString.SuperBuffer+  c-sources: cbits/superbuffer.c++tests:+  spec:+    cpp-options: -DTest+    main: Test.hs+    source-dirs: test+    dependencies:+      - HTF < 0.14+      - QuickCheck < 2.10+      - superbuffer+    ghc-options: -funfolding-use-threshold=16 -O2 -optc-Ofast++benchmarks:+  sbuf-bench:+    main: Bench.hs+    source-dirs: bench+    dependencies:+      - criterion < 1.2+      - superbuffer+    ghc-options: -funfolding-use-threshold=16 -O2 -optc-Ofast
+ src/Data/ByteString/SuperBuffer.hs view
@@ -0,0 +1,85 @@+{-# LANGUAGE EmptyDataDecls #-}+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE ScopedTypeVariables #-}+module Data.ByteString.SuperBuffer+    ( SuperBuffer, withBuffer, appendBuffer+    )+where++import Foreign+import Foreign.C+import Control.Exception++import Data.Coerce+import qualified Data.ByteString as BS+import qualified Data.ByteString.Unsafe as BS++-- | The buffer. Internally only a pointer to a C struct. Don't worry,+-- this module attempts to make usage of the SuperBuffer as safe as possible  in+-- terms of memory leaks (with exceptions).+newtype SuperBuffer+    = SuperBuffer SuperBufferP++-- | Allocate a new buffer with a given initial size. The perfect starting point+-- depends on the expected total size and the average size for a single chunk+-- written with 'appendBuffer'. You can always start with 1024 and optimize from+-- there with benchmarks. Please note that the SuperBuffer will no longer be+-- valid after this function terminates, so do NOT pass it to some other+-- thread without waiting for it to finish in the action.+withBuffer :: Int64 -> (SuperBuffer -> IO ()) -> IO BS.ByteString+withBuffer size action =+    bracket (newBuffer size) destroyBuffer $ \buf ->+    do ok <- try (action buf)+       case ok of+         Left (exception :: SomeException) ->+             do destroyBufferContents buf+                throwIO exception+         Right () ->+             readBuffer buf -- if something goes to shit here, we could be in trouble...+{-# INLINE withBuffer #-}++newBuffer :: Int64 -> IO SuperBuffer+newBuffer size = SuperBuffer <$> new_sbuf (fromIntegral size)+{-# INLINE newBuffer #-}+++-- | Write a bytestring to the buffer and grow the buffer if needed. Note that only+-- one thread at any given time may call this function, so if you are sharing the+-- 'SuperBuffer' between threads make sure you place some type of guarding/locking around+-- this function.+appendBuffer :: SuperBuffer -> BS.ByteString -> IO ()+appendBuffer (SuperBuffer ptr) bs =+    BS.unsafeUseAsCStringLen bs $ \(cstr, len) ->+    append_sbuf ptr cstr (fromIntegral len)+{-# INLINE appendBuffer #-}++destroyBuffer :: SuperBuffer -> IO ()+destroyBuffer (SuperBuffer ptr) = destroy_sbuf ptr+{-# INLINE destroyBuffer #-}++destroyBufferContents :: SuperBuffer -> IO ()+destroyBufferContents (SuperBuffer ptr) = destroyContents_sbuf ptr+{-# INLINE destroyBufferContents #-}++-- | Read the final buffer contents. This must only+-- be called once+readBuffer :: SuperBuffer -> IO BS.ByteString+readBuffer (SuperBuffer ptr) =+    do (cstr, size) <- readLocal+       BS.unsafePackCStringFinalizer (coerce cstr) (fromIntegral size) (free cstr)+    where+      readLocal =+          alloca $ \sizePtr ->+          do cstr <- read_sbuf ptr sizePtr+             size <- peek sizePtr+             pure (cstr, size)+{-# INLINE readBuffer #-}++data SBuf+type SuperBufferP = Ptr SBuf++foreign import ccall unsafe "new_sbuf" new_sbuf :: CSize -> IO SuperBufferP+foreign import ccall unsafe "append_sbuf" append_sbuf :: SuperBufferP -> CString -> CSize -> IO ()+foreign import ccall unsafe "read_sbuf" read_sbuf :: SuperBufferP -> Ptr CSize -> IO CString+foreign import ccall unsafe "destroy_sbuf" destroy_sbuf :: SuperBufferP -> IO ()+foreign import ccall unsafe "destroyContents_sbuf" destroyContents_sbuf :: SuperBufferP -> IO ()
+ stack.yaml view
@@ -0,0 +1,66 @@+# This file was automatically generated by 'stack init'+# +# Some commonly used options have been documented as comments in this file.+# For advanced use and comprehensive documentation of the format, please see:+# http://docs.haskellstack.org/en/stable/yaml_configuration/++# Resolver to choose a 'specific' stackage snapshot or a compiler version.+# A snapshot resolver dictates the compiler version and the set of packages+# to be used for project dependencies. For example:+# +# resolver: lts-3.5+# resolver: nightly-2015-09-21+# resolver: ghc-7.10.2+# resolver: ghcjs-0.1.0_ghc-7.10.2+# resolver:+#  name: custom-snapshot+#  location: "./custom-snapshot.yaml"+resolver: lts-7.11++# User packages to be built.+# Various formats can be used as shown in the example below.+# +# packages:+# - some-directory+# - https://example.com/foo/bar/baz-0.0.2.tar.gz+# - location:+#    git: https://github.com/commercialhaskell/stack.git+#    commit: e7b331f14bcffb8367cd58fbfc8b40ec7642100a+# - location: https://github.com/commercialhaskell/stack/commit/e7b331f14bcffb8367cd58fbfc8b40ec7642100a+#   extra-dep: true+#  subdirs:+#  - auto-update+#  - wai+# +# A package marked 'extra-dep: true' will only be built if demanded by a+# non-dependency (i.e. a user package), and its test suites and benchmarks+# will not be run. This is useful for tweaking upstream packages.+packages:+- '.'+# Dependency packages to be pulled from upstream that are not in the resolver+# (e.g., acme-missiles-0.3)+extra-deps: []++# Override default flag values for local packages and extra-deps+flags: {}++# Extra package databases containing global packages+extra-package-dbs: []++# Control whether we use the GHC we find on the path+# system-ghc: true+# +# Require a specific version of stack, using version ranges+# require-stack-version: -any # Default+# require-stack-version: ">=1.1"+# +# Override the architecture used by stack, especially useful on Windows+# arch: i386+# arch: x86_64+# +# Extra directories used by stack for building+# extra-include-dirs: [/path/to/dir]+# extra-lib-dirs: [/path/to/dir]+# +# Allow a newer minor version of GHC than the snapshot specifies+# compiler-check: newer-minor
+ superbuffer.cabal view
@@ -0,0 +1,65 @@+-- This file has been generated from package.yaml by hpack version 0.14.0.+--+-- see: https://github.com/sol/hpack++name:           superbuffer+version:        0.1.0.0+synopsis:       Efficiently build a bytestring from smaller chunks+description:    Efficiently (both fast and memory efficient) build a bytestring from smaller chunks+category:       Web+homepage:       https://github.com/agrafix/superbuffer#readme+author:         Alexander Thiemann+maintainer:     mail@athiemann.net+copyright:      2016 Alexander Thiemann <mail@athiemann.net>+license:        BSD3+license-file:   LICENSE+build-type:     Simple+cabal-version:  >= 1.10++extra-source-files:+    package.yaml+    README.md+    stack.yaml++library+  hs-source-dirs:+      src+  ghc-options: -Wall+  c-sources:+      cbits/superbuffer.c+  build-depends:+      base >= 4.7 && < 5+    , bytestring < 0.11+  exposed-modules:+      Data.ByteString.SuperBuffer+  other-modules:+      Paths_superbuffer+  default-language: Haskell2010++test-suite spec+  type: exitcode-stdio-1.0+  main-is: Test.hs+  hs-source-dirs:+      test+  ghc-options: -Wall -funfolding-use-threshold=16 -O2 -optc-Ofast+  cpp-options: -DTest+  build-depends:+      base >= 4.7 && < 5+    , bytestring < 0.11+    , HTF < 0.14+    , QuickCheck < 2.10+    , superbuffer+  default-language: Haskell2010++benchmark sbuf-bench+  type: exitcode-stdio-1.0+  main-is: Bench.hs+  hs-source-dirs:+      bench+  ghc-options: -Wall -funfolding-use-threshold=16 -O2 -optc-Ofast+  build-depends:+      base >= 4.7 && < 5+    , bytestring < 0.11+    , criterion < 1.2+    , superbuffer+  default-language: Haskell2010
+ test/Test.hs view
@@ -0,0 +1,54 @@+{-# OPTIONS_GHC -F -pgmF htfpp #-}+{-# LANGUAGE OverloadedStrings #-}+module Main where++import Control.Monad+import Data.ByteString.SuperBuffer+import Data.Int+import qualified Data.ByteString as BS++import Test.Framework+import Test.QuickCheck.Monadic++main :: IO ()+main = htfMain htf_thisModulesTests++test_basic :: IO ()+test_basic =+    do bs <- fillBuf+       assertEqual bs expected+    where+      expected =+          "hello world! Welcome to S U P E R B U F F E R"+      fillBuf =+          withBuffer 8 $ \buf ->+          do appendBuffer buf "hello"+             appendBuffer buf " world"+             appendBuffer buf "!"+             appendBuffer buf " Welcome"+             appendBuffer buf " to"+             appendBuffer buf " S U P E R B U F F E R"++newtype BufferChunks+    = BufferChunks { unBufferChunks :: (Int64, [BS.ByteString]) }+    deriving (Show, Eq)++instance Arbitrary BufferChunks where+    arbitrary = -- 5000 * 200 000 = 1 GB max+        do listSize <- choose (1, 5000)+           chunks <-+               replicateM listSize $+               do bsSize <- choose (0, 200000)+                  pure $ BS.replicate bsSize 84+           bufSize <- choose (1, 1024 * 1024 * 1024)+           pure $ BufferChunks (bufSize, chunks)++prop_appendingWorks :: BufferChunks -> Property+prop_appendingWorks (BufferChunks (bufSize, chunks)) =+    monadicIO $+    do out <- run chunkAction+       assert $ out == BS.concat chunks+    where+      chunkAction =+          withBuffer bufSize $ \buf ->+          forM_ chunks $ appendBuffer buf