csv-enumerator 0.8 → 0.8.2

raw patch · 6 files changed

+168/−8 lines, 6 filesdep ~attoparsecPVP ok

version bump matches the API change (PVP)

Dependency ranges changed: attoparsec

API changes (from Hackage documentation)

Files

• README.markdown +2/−1
• csv-enumerator.cabal +38/−5
• src/Data/CSV/Enumerator.hs +20/−2
• test/Test.hs +57/−0
• test/csv-enumerator-test.cabal +47/−0
• test/test.csv +4/−0

@@ -15,7 +15,7 @@ * Fast operation * Convenient interface that supports a variety of use cases -This library is an attempt to close this gap.+This library is an attempt to close these gaps.   ## This package@@ -88,6 +88,7 @@ * The CSVeable typeclass can be refactored to have a more minimal definition. * Operating on Text in addition to ByteString would be phenomenal. * A test-suite needs to be added.+* Some benchmarking would be nice.   Any and all kinds of help is much appreciated!

@@ -1,5 +1,5 @@ Name:                csv-enumerator-Version:             0.8+Version:             0.8.2 Synopsis:            A flexible, fast, enumerator-based CSV parser library for Haskell.   Homepage:            http://github.com/ozataman/csv-enumerator License:             BSD3@@ -10,15 +10,48 @@ Build-type:          Simple Cabal-version:       >=1.2 Description:-  For more information and examples, check out the README at:-  <http://github.com/ozataman/csv-enumerator>.++  CSV files are the de-facto standard in many situations involving data transfer,+  particularly when dealing with enterprise application or disparate database+  systems.++  .++  While there are a number of CSV libraries in Haskell, at the time of this+  project's start in 2010, there wasn't one that provided all of the following:++  .++  * Full flexibility in quote characters, separators, input/output+  .+  * Constant space operation+  .+  * Robust parsing, correctness and error resiliency+  .+  * Convenient interface that supports a variety of use cases+  .+  * Fast operation+  .++  This library is an attempt to close these gaps.+  .++  For more documentation and examples, check out the README at:+  .++  <http://github.com/ozataman/csv-enumerator>+  .      The API is fairly well documented and I would encourage you to keep your-  haddocks handy.+  haddocks handy. If you run into problems, just email me or holler over at+  #haskell.    extra-source-files:   README.markdown+  test/csv-enumerator-test.cabal+  test/test.csv+  test/Test.hs  Library   hs-source-dirs: src@@ -27,7 +60,7 @@   Other-modules:       Data.CSV.Enumerator.Types   build-depends:-      attoparsec >= 0.8 && < 0.9+      attoparsec >= 0.8 && < 0.10     , attoparsec-enumerator >= 0.2.0.3     , base >= 4 && < 5     , containers >= 0.3

@@ -36,6 +36,7 @@   -- * Other Utilities   , outputRow   , outputRows+  , outputColumns   , writeHeaders   ) @@ -47,6 +48,7 @@ import Control.Monad.IO.Class (liftIO, MonadIO) import qualified Data.ByteString as B import qualified Data.ByteString.Char8 as B8+import Data.ByteString.Char8 (ByteString) import Data.ByteString.Internal (c2w) import qualified Data.Map as M import System.Directory@@ -131,9 +133,9 @@     let        sep = B.pack [c2w (csvOutputColSep s)]        wrapField !f = case (csvOutputQuoteChar s) of-        Just !x -> let qt = c2w x-                  in qt `B.cons` f `B.snoc` qt+        Just !x -> x `B8.cons` escape x f `B8.snoc` x         otherwise -> f+      escape c str = B8.intercalate (B8.pack [c,c]) $ B8.split c str     in B.intercalate sep . map wrapField $ r      fileHeaders _ = Nothing@@ -355,6 +357,21 @@ outputRows s oh = mapM_ (outputRow s oh)  +-- | Expand or contract the given 'MapRow' to contain exactly the given set of+-- columns and then write the row into the given 'Handle'.+--+-- This is helpful in filtering the columns or perhaps combining a number of+-- files that don't have the same columns. +--+-- Missing columns will be left empty.+outputColumns :: CSVSettings -> Handle -> [ByteString] -> MapRow -> IO ()+outputColumns s h cs r = outputRow s h r'+  where+    r' = M.fromList $ map formCol cs+    formCol x = (x, maybe "" id $ M.lookup x r)+++ writeHeaders :: CSVeable r => CSVSettings -> Handle -> [r] -> IO () writeHeaders s h rs = case fileHeaders rs of   Just hs -> B.hPutStrLn h . rowToStr s $ hs@@ -380,6 +397,7 @@ -- Use this datatype when developing iteratees for use with fold* family of -- functions (Row enumarators). data (CSVeable r) => ParsedRow r = ParsedRow (Maybe r) | EOF+  ------------------------------------------------------------------------------ -- | An iteratee that processes each row of a CSV file and updates the

@@ -0,0 +1,57 @@++module Main where++import qualified Data.ByteString.Char8 as B+import Data.Map ((!))+import System.Directory++import Test.Framework (defaultMain, testGroup)+import Test.Framework.Providers.HUnit+import Test.Framework.Providers.QuickCheck2 (testProperty)++import Test.QuickCheck+import Test.HUnit+++import Data.CSV.Enumerator+++main = defaultMain tests++tests = [ testGroup "Basic Ops" baseTests ]+++baseTests = [+    testCase "mapping with id works" test_identityMap+  , testCase "simple parsing works" test_simpleParse+  ]+++test_identityMap = do+  Right r <- mapCSVFile "test.csv" csvSettings f "testOut.csv"+  3 @=? r+  f1 <- readFile "test.csv"+  f2 <- readFile "testOut.csv"+  f1 @=? f2+  removeFile "testOut.csv"+  where +    f :: MapRow -> [MapRow]+    f = return+++test_simpleParse = do+  Right (d :: [MapRow]) <- readCSVFile csvSettings "test.csv" +  mapM_ assertRow d+  where+    assertRow r = v3 @=? (v1 + v2)+      where v1 = readBS $ r ! "Col2"+            v2 = readBS $ r ! "Col3" +            v3 = readBS $ r ! "Sum" +++csvSettings = +  defCSVSettings { csvQuoteChar = Just '`'+                 , csvOutputQuoteChar = Just '`' }+++readBS = read . B.unpack

@@ -0,0 +1,47 @@+Name:                csv-enumerator-test+Version:             0.8.2+Synopsis:            A flexible, fast, enumerator-based CSV parser library for Haskell.  +Homepage:            http://github.com/ozataman/csv-enumerator+License:             BSD3+License-file:        LICENSE+Author:              Ozgun Ataman +Maintainer:          Ozgun Ataman <ozataman@gmail.com>+Category:            Data+Build-type:          Simple+Cabal-version:       >=1.2+Description:+  For more information and examples, check out the README at:+  <http://github.com/ozataman/csv-enumerator>.+  +  The API is fairly well documented and I would encourage you to keep your+  haddocks handy.+++extra-source-files:+  README.markdown++Executable test+  main-is: Test.hs+  hs-source-dirs: ./ ../src+  Other-modules:+      Data.CSV.Enumerator.Types+      Data.CSV.Enumerator+  build-depends:+      attoparsec >= 0.8 && < 0.10+    , attoparsec-enumerator >= 0.2.0.3+    , base >= 4 && < 5+    , containers >= 0.3+    , directory+    , bytestring+    , enumerator >= 0.4.5 && < 0.5+    , transformers >= 0.2+    , safe+    , unix-compat >= 0.2.1.1+    , test-framework+    , test-framework-quickcheck2+    , test-framework-hunit+    , QuickCheck >= 2+    , HUnit >= 1.2+  extensions:+    ScopedTypeVariables+    OverloadedStrings

@@ -0,0 +1,4 @@+`Col1`,`Col2`,`Col3`,`Sum`+`A`,`2`,`3`,`5`+`B`,`3`,`4`,`7`+`Field using the quote char ``this is the in-quoted value```,`4`,`5`,`9`