Frames-streamly (empty) → 0.1.0.0
raw patch · 10 files changed
+1584/−0 lines, 10 filesdep +Framesdep +Frames-streamlydep +base
Dependencies added: Frames, Frames-streamly, base, exceptions, primitive, streamly, text, vinyl
Files
- Changelog.md +9/−0
- Frames-streamly.cabal +68/−0
- LICENSE +30/−0
- Readme.md +99/−0
- example_data/forestfires.csv +518/−0
- examples/DemoPaths.hs +14/−0
- examples/Main.hs +40/−0
- src/Frames/Streamly/CSV.hs +599/−0
- src/Frames/Streamly/InCore.hs +114/−0
- src/Frames/Streamly/Transform.hs +93/−0
+ Changelog.md view
@@ -0,0 +1,9 @@+v0.2.0.1+* Upgraded version bounds++v0.2.0.0+* Added Frames.Constraints with some constraint applying utilities+* Added transformMaybe to Frames.Transform++v0.2.0.1 +* Initial logged version
+ Frames-streamly.cabal view
@@ -0,0 +1,68 @@+cabal-version: 1.12++-- This file has been generated from package.yaml by hpack version 0.31.2.+--+-- see: https://github.com/sol/hpack+--+-- hash: 08f10e29d60a8b555f4bca1c0904ac85d2b35f05bfc03bf2b478af236dfb11db++name: Frames-streamly+version: 0.1.0.0+synopsis: A streamly layer for Frames I/O+description: More information is available in the <https://github.com/adamConnerSax/Frames-streamly/blob/master/Readme.md readme>.'+category: Data+homepage: https://github.com/adamConnerSax/Frames-streamly#readme+bug-reports: https://github.com/adamConnerSax/Frames-streamly/issues+author: Adam Conner-Sax+maintainer: adam_conner_sax@yahoo.com+copyright: 2020 Adam Conner-Sax+license: BSD3+license-file: LICENSE+build-type: Simple+extra-source-files:+ Changelog.md+ Readme.md+data-files:+ forestfires.csv+data-dir: example_data++source-repository head+ type: git+ location: https://github.com/adamConnerSax/Frames-streamly++library+ exposed-modules:+ Paths_Frames_streamly+ Frames.Streamly.CSV+ Frames.Streamly.InCore+ Frames.Streamly.Transform+ hs-source-dirs:+ src+ ghc-options: -Wall -O2 -fdicts-strict -fmax-worker-args=16 -fspec-constr-recursive=16+ build-depends:+ Frames >=0.6 && <0.8+ , base >=4.12.0 && <4.15+ , exceptions >=0.10.0 && <0.11+ , primitive >=0.7 && <0.8+ , streamly >=0.7 && <0.8+ , text >=1.2.3 && <1.3+ , vinyl >=0.12 && <0.14+ default-language: Haskell2010++test-suite Demo+ type: exitcode-stdio-1.0+ main-is: Main.hs+ other-modules:+ DemoPaths+ Paths_Frames_streamly+ hs-source-dirs:+ examples+ ghc-options: -Wall -O2 -fdicts-strict -fmax-worker-args=16 -fspec-constr-recursive=16+ build-depends:+ Frames+ , Frames-streamly+ , base+ , streamly+ , text+ , vinyl+ default-language: Haskell2010
+ LICENSE view
@@ -0,0 +1,30 @@+Copyright (c) 2018, Adam Conner-Sax++All rights reserved.++Redistribution and use in source and binary forms, with or without+modification, are permitted provided that the following conditions are met:++ * Redistributions of source code must retain the above copyright+ notice, this list of conditions and the following disclaimer.++ * Redistributions in binary form must reproduce the above+ copyright notice, this list of conditions and the following+ disclaimer in the documentation and/or other materials provided+ with the distribution.++ * Neither the name of Adam Conner-Sax nor the names of other+ contributors may be used to endorse or promote products derived+ from this software without specific prior written permission.++THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ Readme.md view
@@ -0,0 +1,99 @@+# Frames-streamly- v 0.1.0.0++[![Build Status][travis-badge]][travis]+[![Hackage][hackage-badge]][hackage]+[![Hackage Dependencies][hackage-deps-badge]][hackage-deps]++This library contains some useful functions for using the +[Frames](https://hackage.haskell.org/package/Frames) +package with [streamly](https://hackage.haskell.org/package/streamly).++Frames has some built-in dependencies on the +[Pipes](https://hackage.haskell.org/package/pipes) package,+a few of which--primarily file I/O-- require users of Frames to use+the Pipes package explicitly. Streamly provides much of the same +functionality as Pipes and may be some users preferred streaming +interface.++This package replicates all the external-facing bits of Frames that+rely on Pipes and uses streamly instead. It also fleshes out the Frames+API in a couple of places:++1. It adds some flexibility to the functions to write CSV files. +Frames supported formatting of fields for CSV +via a typeclass ```ShowCSV```. That is supported here as well. But+this package also supports using the ```Show``` instance, and, for more+customized formatting,+creating field-by-field formatting on the fly via+a [Vinyl](https://hackage.haskell.org/package/vinyl) record of functions.+Helpful combinators are provided for formatting any single field with+a ```Show``` instance or a ```ShowCSV``` instance or a user provided+function from the field type to ```Text```. ++2. It adds some (experimental) support for ```Frame``` transformations +using Streamly streams as an intermediate state for transformations+which may benefit from the concurrency available in streamly. Such+transformations first make any foldable of Records +(including a ```Frame```) into a stream, apply a streamly transformation+to a stream of some other records and then transforms those into a frame.+So the result is a frame -> frame function but one that can take advantage+of streamly's features at the cost of the transformation into a stream and then+back in to a Frame.+This allows use of the concurrent features of +Streamly for functions like ```mapM``` or ```mapMaybeM```. ++3. It adds Streamly folds for the various stream to in-core +transformations in case users want to use them directly +in stream to Frame transformations. Frames exposed only the functions+to transform an entire stream (a pipe producer) into Frames "AoS" +structure. This library provides that functionality as well, in this case+using streamly streams as the input.+But here we also expose streamly folds from streams of+Records to Frames so that more complex stream to Frame transformations+can be done by the user. For example, suppose you are doing+a map/reduce on a large data set and you want to store the +grouped subsets as Frames for memory-efficiency. These folds+make that simpler.+++An example using some of the utilities is [here](https://github.com/adamConnerSax/Frames-streamly/blob/master/examples/Main.hs).+_______++LICENSE (BSD-3-Clause)+_______+Copyright (c) 2020, Adam Conner-Sax, All rights reserved.++Redistribution and use in source and binary forms, with or without+modification, are permitted provided that the following conditions are met:++ * Redistributions of source code must retain the above copyright+ notice, this list of conditions and the following disclaimer.++ * Redistributions in binary form must reproduce the above+ copyright notice, this list of conditions and the following+ disclaimer in the documentation and/or other materials provided+ with the distribution.++ * Neither the name of Adam Conner-Sax nor the names of other+ contributors may be used to endorse or promote products derived+ from this software without specific prior written permission.++THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.+++[travis]: <https://travis-ci.org/adamConnerSax/Frames-streamly>+[travis-badge]: <https://travis-ci.org/adamConnerSax/Frames-streamly.svg?branch=master>+[hackage]: <https://hackage.haskell.org/package/Frames-streamly>+[hackage-badge]: <https://img.shields.io/hackage/v/Frames-streamly.svg>+[hackage-deps-badge]: <https://img.shields.io/hackage-deps/v/Frames-streamly.svg>+[hackage-deps]: <http://packdeps.haskellers.com/feed?needle=Frames-streamly>
+ example_data/forestfires.csv view
@@ -0,0 +1,518 @@+X,Y,month,day,FFMC,DMC,DC,ISI,temp,RH,wind,rain,area+7,5,mar,fri,86.2,26.2,94.3,5.1,8.2,51,6.7,0,0+7,4,oct,tue,90.6,35.4,669.1,6.7,18,33,0.9,0,0+7,4,oct,sat,90.6,43.7,686.9,6.7,14.6,33,1.3,0,0+8,6,mar,fri,91.7,33.3,77.5,9,8.3,97,4,0.2,0+8,6,mar,sun,89.3,51.3,102.2,9.6,11.4,99,1.8,0,0+8,6,aug,sun,92.3,85.3,488,14.7,22.2,29,5.4,0,0+8,6,aug,mon,92.3,88.9,495.6,8.5,24.1,27,3.1,0,0+8,6,aug,mon,91.5,145.4,608.2,10.7,8,86,2.2,0,0+8,6,sep,tue,91,129.5,692.6,7,13.1,63,5.4,0,0+7,5,sep,sat,92.5,88,698.6,7.1,22.8,40,4,0,0+7,5,sep,sat,92.5,88,698.6,7.1,17.8,51,7.2,0,0+7,5,sep,sat,92.8,73.2,713,22.6,19.3,38,4,0,0+6,5,aug,fri,63.5,70.8,665.3,0.8,17,72,6.7,0,0+6,5,sep,mon,90.9,126.5,686.5,7,21.3,42,2.2,0,0+6,5,sep,wed,92.9,133.3,699.6,9.2,26.4,21,4.5,0,0+6,5,sep,fri,93.3,141.2,713.9,13.9,22.9,44,5.4,0,0+5,5,mar,sat,91.7,35.8,80.8,7.8,15.1,27,5.4,0,0+8,5,oct,mon,84.9,32.8,664.2,3,16.7,47,4.9,0,0+6,4,mar,wed,89.2,27.9,70.8,6.3,15.9,35,4,0,0+6,4,apr,sat,86.3,27.4,97.1,5.1,9.3,44,4.5,0,0+6,4,sep,tue,91,129.5,692.6,7,18.3,40,2.7,0,0+5,4,sep,mon,91.8,78.5,724.3,9.2,19.1,38,2.7,0,0+7,4,jun,sun,94.3,96.3,200,56.1,21,44,4.5,0,0+7,4,aug,sat,90.2,110.9,537.4,6.2,19.5,43,5.8,0,0+7,4,aug,sat,93.5,139.4,594.2,20.3,23.7,32,5.8,0,0+7,4,aug,sun,91.4,142.4,601.4,10.6,16.3,60,5.4,0,0+7,4,sep,fri,92.4,117.9,668,12.2,19,34,5.8,0,0+7,4,sep,mon,90.9,126.5,686.5,7,19.4,48,1.3,0,0+6,3,sep,sat,93.4,145.4,721.4,8.1,30.2,24,2.7,0,0+6,3,sep,sun,93.5,149.3,728.6,8.1,22.8,39,3.6,0,0+6,3,sep,fri,94.3,85.1,692.3,15.9,25.4,24,3.6,0,0+6,3,sep,mon,88.6,91.8,709.9,7.1,11.2,78,7.6,0,0+6,3,sep,fri,88.6,69.7,706.8,5.8,20.6,37,1.8,0,0+6,3,sep,sun,91.7,75.6,718.3,7.8,17.7,39,3.6,0,0+6,3,sep,mon,91.8,78.5,724.3,9.2,21.2,32,2.7,0,0+6,3,sep,tue,90.3,80.7,730.2,6.3,18.2,62,4.5,0,0+6,3,oct,tue,90.6,35.4,669.1,6.7,21.7,24,4.5,0,0+7,4,oct,fri,90,41.5,682.6,8.7,11.3,60,5.4,0,0+7,3,oct,sat,90.6,43.7,686.9,6.7,17.8,27,4,0,0+4,4,mar,tue,88.1,25.7,67.6,3.8,14.1,43,2.7,0,0+4,4,jul,tue,79.5,60.6,366.7,1.5,23.3,37,3.1,0,0+4,4,aug,sat,90.2,96.9,624.2,8.9,18.4,42,6.7,0,0+4,4,aug,tue,94.8,108.3,647.1,17,16.6,54,5.4,0,0+4,4,sep,sat,92.5,88,698.6,7.1,19.6,48,2.7,0,0+4,4,sep,wed,90.1,82.9,735.7,6.2,12.9,74,4.9,0,0+5,6,sep,wed,94.3,85.1,692.3,15.9,25.9,24,4,0,0+5,6,sep,mon,90.9,126.5,686.5,7,14.7,70,3.6,0,0+6,6,jul,mon,94.2,62.3,442.9,11,23,36,3.1,0,0+4,4,mar,mon,87.2,23.9,64.7,4.1,11.8,35,1.8,0,0+4,4,mar,mon,87.6,52.2,103.8,5,11,46,5.8,0,0+4,4,sep,thu,92.9,137,706.4,9.2,20.8,17,1.3,0,0+4,3,aug,sun,90.2,99.6,631.2,6.3,21.5,34,2.2,0,0+4,3,aug,wed,92.1,111.2,654.1,9.6,20.4,42,4.9,0,0+4,3,aug,wed,92.1,111.2,654.1,9.6,20.4,42,4.9,0,0+4,3,aug,thu,91.7,114.3,661.3,6.3,17.6,45,3.6,0,0+4,3,sep,thu,92.9,137,706.4,9.2,27.7,24,2.2,0,0+4,3,sep,tue,90.3,80.7,730.2,6.3,17.8,63,4.9,0,0+4,3,oct,sun,92.6,46.5,691.8,8.8,13.8,50,2.7,0,0+2,2,feb,mon,84,9.3,34,2.1,13.9,40,5.4,0,0+2,2,feb,fri,86.6,13.2,43,5.3,12.3,51,0.9,0,0+2,2,mar,sun,89.3,51.3,102.2,9.6,11.5,39,5.8,0,0+2,2,mar,sun,89.3,51.3,102.2,9.6,5.5,59,6.3,0,0+2,2,aug,thu,93,75.3,466.6,7.7,18.8,35,4.9,0,0+2,2,aug,sun,90.2,99.6,631.2,6.3,20.8,33,2.7,0,0+2,2,aug,mon,91.1,103.2,638.8,5.8,23.1,31,3.1,0,0+2,2,aug,thu,91.7,114.3,661.3,6.3,18.6,44,4.5,0,0+2,2,sep,fri,92.4,117.9,668,12.2,23,37,4.5,0,0+2,2,sep,fri,92.4,117.9,668,12.2,19.6,33,5.4,0,0+2,2,sep,fri,92.4,117.9,668,12.2,19.6,33,6.3,0,0+4,5,mar,fri,91.7,33.3,77.5,9,17.2,26,4.5,0,0+4,5,mar,fri,91.2,48.3,97.8,12.5,15.8,27,7.6,0,0+4,5,sep,fri,94.3,85.1,692.3,15.9,17.7,37,3.6,0,0+5,4,mar,fri,91.7,33.3,77.5,9,15.6,25,6.3,0,0+5,4,aug,tue,88.8,147.3,614.5,9,17.3,43,4.5,0,0+5,4,sep,fri,93.3,141.2,713.9,13.9,27.6,30,1.3,0,0+9,9,feb,thu,84.2,6.8,26.6,7.7,6.7,79,3.1,0,0+9,9,feb,fri,86.6,13.2,43,5.3,15.7,43,3.1,0,0+1,3,mar,mon,87.6,52.2,103.8,5,8.3,72,3.1,0,0+1,2,aug,fri,90.1,108,529.8,12.5,14.7,66,2.7,0,0+1,2,aug,tue,91,121.2,561.6,7,21.6,19,6.7,0,0+1,2,aug,sun,91.4,142.4,601.4,10.6,19.5,39,6.3,0,0+1,2,aug,sun,90.2,99.6,631.2,6.3,17.9,44,2.2,0,0+1,2,aug,tue,94.8,108.3,647.1,17,18.6,51,4.5,0,0+1,2,aug,wed,92.1,111.2,654.1,9.6,16.6,47,0.9,0,0+1,2,aug,thu,91.7,114.3,661.3,6.3,20.2,45,3.6,0,0+1,2,sep,thu,92.9,137,706.4,9.2,21.5,15,0.9,0,0+1,2,sep,thu,92.9,137,706.4,9.2,25.4,27,2.2,0,0+1,2,sep,thu,92.9,137,706.4,9.2,22.4,34,2.2,0,0+1,2,sep,sun,93.5,149.3,728.6,8.1,25.3,36,3.6,0,0+6,5,mar,sat,91.7,35.8,80.8,7.8,17.4,25,4.9,0,0+6,5,aug,sat,90.2,96.9,624.2,8.9,14.7,59,5.8,0,0+8,6,mar,fri,91.7,35.8,80.8,7.8,17.4,24,5.4,0,0+8,6,aug,sun,92.3,85.3,488,14.7,20.8,32,6.3,0,0+8,6,aug,sun,91.4,142.4,601.4,10.6,18.2,43,4.9,0,0+8,6,aug,mon,91.1,103.2,638.8,5.8,23.4,22,2.7,0,0+4,4,sep,sun,89.7,90,704.4,4.8,17.8,64,1.3,0,0+3,4,feb,sat,83.9,8,30.2,2.6,12.7,48,1.8,0,0+3,4,mar,sat,69,2.4,15.5,0.7,17.4,24,5.4,0,0+3,4,aug,sun,91.4,142.4,601.4,10.6,11.6,87,4.5,0,0+3,4,aug,sun,91.4,142.4,601.4,10.6,19.8,39,5.4,0,0+3,4,aug,sun,91.4,142.4,601.4,10.6,19.8,39,5.4,0,0+3,4,aug,tue,88.8,147.3,614.5,9,14.4,66,5.4,0,0+2,4,aug,tue,94.8,108.3,647.1,17,20.1,40,4,0,0+2,4,sep,sat,92.5,121.1,674.4,8.6,24.1,29,4.5,0,0+2,4,jan,sat,82.1,3.7,9.3,2.9,5.3,78,3.1,0,0+4,5,mar,fri,85.9,19.5,57.3,2.8,12.7,52,6.3,0,0+4,5,mar,thu,91.4,30.7,74.3,7.5,18.2,29,3.1,0,0+4,5,aug,sun,90.2,99.6,631.2,6.3,21.4,33,3.1,0,0+4,5,sep,sat,92.5,88,698.6,7.1,20.3,45,3.1,0,0+4,5,sep,mon,88.6,91.8,709.9,7.1,17.4,56,5.4,0,0+4,4,mar,fri,85.9,19.5,57.3,2.8,13.7,43,5.8,0,0+3,4,mar,fri,91.7,33.3,77.5,9,18.8,18,4.5,0,0+3,4,sep,sun,89.7,90,704.4,4.8,22.8,39,3.6,0,0+3,4,sep,mon,91.8,78.5,724.3,9.2,18.9,35,2.7,0,0+3,4,mar,tue,88.1,25.7,67.6,3.8,15.8,27,7.6,0,0+3,5,mar,tue,88.1,25.7,67.6,3.8,15.5,27,6.3,0,0+3,4,mar,sat,91.7,35.8,80.8,7.8,11.6,30,6.3,0,0+3,4,mar,sat,91.7,35.8,80.8,7.8,15.2,27,4.9,0,0+3,4,mar,mon,90.1,39.7,86.6,6.2,10.6,30,4,0,0+3,4,aug,thu,93,75.3,466.6,7.7,19.6,36,3.1,0,0+3,4,aug,mon,91.5,145.4,608.2,10.7,10.3,74,2.2,0,0+3,4,aug,mon,91.5,145.4,608.2,10.7,17.1,43,5.4,0,0+3,4,sep,sun,92.4,124.1,680.7,8.5,22.5,42,5.4,0,0+3,4,sep,tue,84.4,73.4,671.9,3.2,17.9,45,3.1,0,0+3,4,sep,fri,94.3,85.1,692.3,15.9,19.8,50,5.4,0,0+3,4,oct,sun,92.6,46.5,691.8,8.8,20.6,24,5.4,0,0+3,5,mar,mon,87.6,52.2,103.8,5,9,49,2.2,0,0+3,5,sep,fri,93.5,149.3,728.6,8.1,17.2,43,3.1,0,0+3,5,oct,wed,91.4,37.9,673.8,5.2,15.9,46,3.6,0,0+2,5,oct,sun,92.6,46.5,691.8,8.8,15.4,35,0.9,0,0+4,6,feb,sat,68.2,21.5,87.2,0.8,15.4,40,2.7,0,0+4,6,mar,mon,87.2,23.9,64.7,4.1,14,39,3.1,0,0+4,6,mar,sun,89.3,51.3,102.2,9.6,10.6,46,4.9,0,0+4,6,sep,thu,93.7,80.9,685.2,17.9,17.6,42,3.1,0,0+3,5,mar,tue,88.1,25.7,67.6,3.8,14.9,38,2.7,0,0+3,5,aug,sat,93.5,139.4,594.2,20.3,17.6,52,5.8,0,0+3,6,sep,sun,92.4,124.1,680.7,8.5,17.2,58,1.3,0,0+3,6,sep,mon,90.9,126.5,686.5,7,15.6,66,3.1,0,0+9,9,jul,tue,85.8,48.3,313.4,3.9,18,42,2.7,0,0.36+1,4,sep,tue,91,129.5,692.6,7,21.7,38,2.2,0,0.43+2,5,sep,mon,90.9,126.5,686.5,7,21.9,39,1.8,0,0.47+1,2,aug,wed,95.5,99.9,513.3,13.2,23.3,31,4.5,0,0.55+8,6,aug,fri,90.1,108,529.8,12.5,21.2,51,8.9,0,0.61+1,2,jul,sat,90,51.3,296.3,8.7,16.6,53,5.4,0,0.71+2,5,aug,wed,95.5,99.9,513.3,13.2,23.8,32,5.4,0,0.77+6,5,aug,thu,95.2,131.7,578.8,10.4,27.4,22,4,0,0.9+5,4,mar,mon,90.1,39.7,86.6,6.2,13.2,40,5.4,0,0.95+8,3,sep,tue,84.4,73.4,671.9,3.2,24.2,28,3.6,0,0.96+2,2,aug,tue,94.8,108.3,647.1,17,17.4,43,6.7,0,1.07+8,6,sep,thu,93.7,80.9,685.2,17.9,23.7,25,4.5,0,1.12+6,5,jun,fri,92.5,56.4,433.3,7.1,23.2,39,5.4,0,1.19+9,9,jul,sun,90.1,68.6,355.2,7.2,24.8,29,2.2,0,1.36+3,4,jul,sat,90.1,51.2,424.1,6.2,24.6,43,1.8,0,1.43+5,4,sep,fri,94.3,85.1,692.3,15.9,20.1,47,4.9,0,1.46+1,5,sep,sat,93.4,145.4,721.4,8.1,29.6,27,2.7,0,1.46+7,4,aug,sun,94.8,108.3,647.1,17,16.4,47,1.3,0,1.56+2,4,sep,sat,93.4,145.4,721.4,8.1,28.6,27,2.2,0,1.61+2,2,aug,wed,92.1,111.2,654.1,9.6,18.4,45,3.6,0,1.63+2,4,aug,wed,92.1,111.2,654.1,9.6,20.5,35,4,0,1.64+7,4,sep,fri,92.4,117.9,668,12.2,19,34,5.8,0,1.69+7,4,mar,mon,90.1,39.7,86.6,6.2,16.1,29,3.1,0,1.75+6,4,aug,thu,95.2,131.7,578.8,10.4,20.3,41,4,0,1.9+6,3,mar,sat,90.6,50.1,100.4,7.8,15.2,31,8.5,0,1.94+8,6,sep,sat,92.5,121.1,674.4,8.6,17.8,56,1.8,0,1.95+8,5,sep,sun,89.7,90,704.4,4.8,17.8,67,2.2,0,2.01+6,5,mar,thu,84.9,18.2,55,3,5.3,70,4.5,0,2.14+6,5,aug,wed,92.1,111.2,654.1,9.6,16.6,47,0.9,0,2.29+6,5,aug,wed,96,127.1,570.5,16.5,23.4,33,4.5,0,2.51+6,5,mar,fri,91.2,48.3,97.8,12.5,14.6,26,9.4,0,2.53+8,6,aug,thu,95.2,131.7,578.8,10.4,20.7,45,2.2,0,2.55+5,4,sep,wed,92.9,133.3,699.6,9.2,21.9,35,1.8,0,2.57+8,6,aug,wed,85.6,90.4,609.6,6.6,17.4,50,4,0,2.69+7,4,aug,sun,91.4,142.4,601.4,10.6,20.1,39,5.4,0,2.74+4,4,sep,mon,90.9,126.5,686.5,7,17.7,39,2.2,0,3.07+1,4,aug,sat,90.2,96.9,624.2,8.9,14.2,53,1.8,0,3.5+1,4,aug,sat,90.2,96.9,624.2,8.9,20.3,39,4.9,0,4.53+6,5,apr,thu,81.5,9.1,55.2,2.7,5.8,54,5.8,0,4.61+2,5,aug,sun,90.2,99.6,631.2,6.3,19.2,44,2.7,0,4.69+2,5,sep,wed,90.1,82.9,735.7,6.2,18.3,45,2.2,0,4.88+8,6,aug,tue,88.8,147.3,614.5,9,14.4,66,5.4,0,5.23+1,3,sep,sun,92.4,124.1,680.7,8.5,23.9,32,6.7,0,5.33+8,6,oct,mon,84.9,32.8,664.2,3,19.1,32,4,0,5.44+5,4,feb,sun,86.8,15.6,48.3,3.9,12.4,53,2.2,0,6.38+7,4,oct,mon,91.7,48.5,696.1,11.1,16.8,45,4.5,0,6.83+8,6,aug,fri,93.9,135.7,586.7,15.1,20.8,34,4.9,0,6.96+2,5,sep,tue,91,129.5,692.6,7,17.6,46,3.1,0,7.04+8,6,mar,sun,89.3,51.3,102.2,9.6,11.5,39,5.8,0,7.19+1,5,sep,mon,90.9,126.5,686.5,7,21,42,2.2,0,7.3+6,4,mar,sat,90.8,41.9,89.4,7.9,13.3,42,0.9,0,7.4+7,4,mar,sun,90.7,44,92.4,5.5,11.5,60,4,0,8.24+6,5,mar,fri,91.2,48.3,97.8,12.5,11.7,33,4,0,8.31+2,5,aug,thu,95.2,131.7,578.8,10.4,24.2,28,2.7,0,8.68+2,2,aug,tue,94.8,108.3,647.1,17,24.6,22,4.5,0,8.71+4,5,sep,wed,92.9,133.3,699.6,9.2,24.3,25,4,0,9.41+2,2,aug,tue,94.8,108.3,647.1,17,24.6,22,4.5,0,10.01+2,5,aug,fri,93.9,135.7,586.7,15.1,23.5,36,5.4,0,10.02+6,5,apr,thu,81.5,9.1,55.2,2.7,5.8,54,5.8,0,10.93+4,5,sep,thu,92.9,137,706.4,9.2,21.5,15,0.9,0,11.06+3,4,sep,tue,91,129.5,692.6,7,13.9,59,6.3,0,11.24+2,4,sep,mon,63.5,70.8,665.3,0.8,22.6,38,3.6,0,11.32+1,5,sep,tue,91,129.5,692.6,7,21.6,33,2.2,0,11.53+6,5,mar,sun,90.1,37.6,83.7,7.2,12.4,54,3.6,0,12.1+7,4,feb,sun,83.9,8.7,32.1,2.1,8.8,68,2.2,0,13.05+8,6,oct,wed,91.4,37.9,673.8,5.2,20.2,37,2.7,0,13.7+5,6,mar,sat,90.6,50.1,100.4,7.8,15.1,64,4,0,13.99+4,5,sep,thu,92.9,137,706.4,9.2,22.1,34,1.8,0,14.57+2,2,aug,sat,93.5,139.4,594.2,20.3,22.9,31,7.2,0,15.45+7,5,sep,tue,91,129.5,692.6,7,20.7,37,2.2,0,17.2+6,5,sep,fri,92.4,117.9,668,12.2,19.6,33,6.3,0,19.23+8,3,sep,thu,93.7,80.9,685.2,17.9,23.2,26,4.9,0,23.41+4,4,oct,sat,90.6,43.7,686.9,6.7,18.4,25,3.1,0,24.23+7,4,aug,sat,93.5,139.4,594.2,20.3,5.1,96,5.8,0,26+7,4,sep,fri,94.3,85.1,692.3,15.9,20.1,47,4.9,0,26.13+7,3,mar,mon,87.6,52.2,103.8,5,11,46,5.8,0,27.35+4,4,mar,sat,91.7,35.8,80.8,7.8,17,27,4.9,0,28.66+4,4,mar,sat,91.7,35.8,80.8,7.8,17,27,4.9,0,28.66+4,4,sep,sun,92.4,124.1,680.7,8.5,16.9,60,1.3,0,29.48+1,3,sep,mon,88.6,91.8,709.9,7.1,12.4,73,6.3,0,30.32+4,5,sep,wed,92.9,133.3,699.6,9.2,19.4,19,1.3,0,31.72+6,5,mar,mon,90.1,39.7,86.6,6.2,15.2,27,3.1,0,31.86+8,6,aug,sun,90.2,99.6,631.2,6.3,16.2,59,3.1,0,32.07+3,4,sep,fri,93.3,141.2,713.9,13.9,18.6,49,3.6,0,35.88+4,3,mar,mon,87.6,52.2,103.8,5,11,46,5.8,0,36.85+2,2,jul,fri,88.3,150.3,309.9,6.8,13.4,79,3.6,0,37.02+7,4,sep,wed,90.1,82.9,735.7,6.2,15.4,57,4.5,0,37.71+4,4,sep,sun,93.5,149.3,728.6,8.1,22.9,39,4.9,0,48.55+7,5,oct,mon,91.7,48.5,696.1,11.1,16.1,44,4,0,49.37+8,6,aug,sat,92.2,81.8,480.8,11.9,20.1,34,4.5,0,58.3+4,6,sep,sun,93.5,149.3,728.6,8.1,28.3,26,3.1,0,64.1+8,6,aug,sat,92.2,81.8,480.8,11.9,16.4,43,4,0,71.3+4,4,sep,wed,92.9,133.3,699.6,9.2,26.4,21,4.5,0,88.49+1,5,sep,sun,93.5,149.3,728.6,8.1,27.8,27,3.1,0,95.18+6,4,sep,tue,91,129.5,692.6,7,18.7,43,2.7,0,103.39+9,4,sep,tue,84.4,73.4,671.9,3.2,24.3,36,3.1,0,105.66+4,5,sep,sat,92.5,121.1,674.4,8.6,17.7,25,3.1,0,154.88+8,6,aug,sun,91.4,142.4,601.4,10.6,19.6,41,5.8,0,196.48+2,2,sep,sat,92.5,121.1,674.4,8.6,18.2,46,1.8,0,200.94+1,2,sep,tue,91,129.5,692.6,7,18.8,40,2.2,0,212.88+6,5,sep,sat,92.5,121.1,674.4,8.6,25.1,27,4,0,1090.84+7,5,apr,sun,81.9,3,7.9,3.5,13.4,75,1.8,0,0+6,3,apr,wed,88,17.2,43.5,3.8,15.2,51,2.7,0,0+4,4,apr,fri,83,23.3,85.3,2.3,16.7,20,3.1,0,0+2,4,aug,sun,94.2,122.3,589.9,12.9,15.4,66,4,0,10.13+7,4,aug,sun,91.8,175.1,700.7,13.8,21.9,73,7.6,1,0+2,4,aug,sun,91.8,175.1,700.7,13.8,22.4,54,7.6,0,2.87+3,4,aug,sun,91.8,175.1,700.7,13.8,26.8,38,6.3,0,0.76+5,4,aug,sun,91.8,175.1,700.7,13.8,25.7,39,5.4,0,0.09+2,4,aug,wed,92.2,91.6,503.6,9.6,20.7,70,2.2,0,0.75+8,6,aug,wed,93.1,157.3,666.7,13.5,28.7,28,2.7,0,0+3,4,aug,wed,93.1,157.3,666.7,13.5,21.7,40,0.4,0,2.47+8,5,aug,wed,93.1,157.3,666.7,13.5,26.8,25,3.1,0,0.68+8,5,aug,wed,93.1,157.3,666.7,13.5,24,36,3.1,0,0.24+6,5,aug,wed,93.1,157.3,666.7,13.5,22.1,37,3.6,0,0.21+7,4,aug,thu,91.9,109.2,565.5,8,21.4,38,2.7,0,1.52+6,3,aug,thu,91.6,138.1,621.7,6.3,18.9,41,3.1,0,10.34+2,5,aug,thu,87.5,77,694.8,5,22.3,46,4,0,0+8,6,aug,sat,94.2,117.2,581.1,11,23.9,41,2.2,0,8.02+4,3,aug,sat,94.2,117.2,581.1,11,21.4,44,2.7,0,0.68+3,4,aug,sat,91.8,170.9,692.3,13.7,20.6,59,0.9,0,0+7,4,aug,sat,91.8,170.9,692.3,13.7,23.7,40,1.8,0,1.38+2,4,aug,mon,93.6,97.9,542,14.4,28.3,32,4,0,8.85+3,4,aug,fri,91.6,112.4,573,8.9,11.2,84,7.6,0,3.3+2,4,aug,fri,91.6,112.4,573,8.9,21.4,42,3.1,0,4.25+6,3,aug,fri,91.1,141.1,629.1,7.1,19.3,39,3.6,0,1.56+4,4,aug,fri,94.3,167.6,684.4,13,21.8,53,3.1,0,6.54+4,4,aug,tue,93.7,102.2,550.3,14.6,22.1,54,7.6,0,0.79+6,5,aug,tue,94.3,131.7,607.1,22.7,19.4,55,4,0,0.17+2,2,aug,tue,92.1,152.6,658.2,14.3,23.7,24,3.1,0,0+3,4,aug,tue,92.1,152.6,658.2,14.3,21,32,3.1,0,0+4,4,aug,tue,92.1,152.6,658.2,14.3,19.1,53,2.7,0,4.4+2,2,aug,tue,92.1,152.6,658.2,14.3,21.8,56,3.1,0,0.52+8,6,aug,tue,92.1,152.6,658.2,14.3,20.1,58,4.5,0,9.27+2,5,aug,tue,92.1,152.6,658.2,14.3,20.2,47,4,0,3.09+4,6,dec,sun,84.4,27.2,353.5,6.8,4.8,57,8.5,0,8.98+8,6,dec,wed,84,27.8,354.6,5.3,5.1,61,8,0,11.19+4,6,dec,thu,84.6,26.4,352,2,5.1,61,4.9,0,5.38+4,4,dec,mon,85.4,25.4,349.7,2.6,4.6,21,8.5,0,17.85+3,4,dec,mon,85.4,25.4,349.7,2.6,4.6,21,8.5,0,10.73+4,4,dec,mon,85.4,25.4,349.7,2.6,4.6,21,8.5,0,22.03+4,4,dec,mon,85.4,25.4,349.7,2.6,4.6,21,8.5,0,9.77+4,6,dec,fri,84.7,26.7,352.6,4.1,2.2,59,4.9,0,9.27+6,5,dec,tue,85.4,25.4,349.7,2.6,5.1,24,8.5,0,24.77+6,3,feb,sun,84.9,27.5,353.5,3.4,4.2,51,4,0,0+3,4,feb,wed,86.9,6.6,18.7,3.2,8.8,35,3.1,0,1.1+5,4,feb,fri,85.2,4.9,15.8,6.3,7.5,46,8,0,24.24+2,5,jul,sun,93.9,169.7,411.8,12.3,23.4,40,6.3,0,0+7,6,jul,wed,91.2,183.1,437.7,12.5,12.6,90,7.6,0.2,0+7,4,jul,sat,91.6,104.2,474.9,9,22.1,49,2.7,0,0+7,4,jul,sat,91.6,104.2,474.9,9,24.2,32,1.8,0,0+7,4,jul,sat,91.6,104.2,474.9,9,24.3,30,1.8,0,0+2,5,jul,sat,91.6,104.2,474.9,9,18.7,53,1.8,0,0+9,4,jul,sat,91.6,104.2,474.9,9,25.3,39,0.9,0,8+4,5,jul,fri,91.6,100.2,466.3,6.3,22.9,40,1.3,0,2.64+7,6,jul,tue,93.1,180.4,430.8,11,26.9,28,5.4,0,86.45+8,6,jul,tue,92.3,88.8,440.9,8.5,17.1,67,3.6,0,6.57+7,5,jun,sun,93.1,180.4,430.8,11,22.2,48,1.3,0,0+6,4,jun,sun,90.4,89.5,290.8,6.4,14.3,46,1.8,0,0.9+8,6,jun,sun,90.4,89.5,290.8,6.4,15.4,45,2.2,0,0+8,6,jun,wed,91.2,147.8,377.2,12.7,19.6,43,4.9,0,0+6,5,jun,sat,53.4,71,233.8,0.4,10.6,90,2.7,0,0+6,5,jun,mon,90.4,93.3,298.1,7.5,20.7,25,4.9,0,0+6,5,jun,mon,90.4,93.3,298.1,7.5,19.1,39,5.4,0,3.52+3,6,jun,fri,91.1,94.1,232.1,7.1,19.2,38,4.5,0,0+3,6,jun,fri,91.1,94.1,232.1,7.1,19.2,38,4.5,0,0+6,5,may,sat,85.1,28,113.8,3.5,11.3,94,4.9,0,0+1,4,sep,sun,89.6,84.1,714.3,5.7,19,52,2.2,0,0+7,4,sep,sun,89.6,84.1,714.3,5.7,17.1,53,5.4,0,0.41+3,4,sep,sun,89.6,84.1,714.3,5.7,23.8,35,3.6,0,5.18+2,4,sep,sun,92.4,105.8,758.1,9.9,16,45,1.8,0,0+2,4,sep,sun,92.4,105.8,758.1,9.9,24.9,27,2.2,0,0+7,4,sep,sun,92.4,105.8,758.1,9.9,25.3,27,2.7,0,0+6,3,sep,sun,92.4,105.8,758.1,9.9,24.8,28,1.8,0,14.29+2,4,sep,sun,50.4,46.2,706.6,0.4,12.2,78,6.3,0,0+6,5,sep,wed,92.6,115.4,777.1,8.8,24.3,27,4.9,0,0+4,4,sep,wed,92.6,115.4,777.1,8.8,19.7,41,1.8,0,1.58+3,4,sep,wed,91.2,134.7,817.5,7.2,18.5,30,2.7,0,0+4,5,sep,thu,92.4,96.2,739.4,8.6,18.6,24,5.8,0,0+4,4,sep,thu,92.4,96.2,739.4,8.6,19.2,24,4.9,0,3.78+6,5,sep,thu,92.8,119,783.5,7.5,21.6,27,2.2,0,0+5,4,sep,thu,92.8,119,783.5,7.5,21.6,28,6.3,0,4.41+6,3,sep,thu,92.8,119,783.5,7.5,18.9,34,7.2,0,34.36+1,4,sep,thu,92.8,119,783.5,7.5,16.8,28,4,0,7.21+6,5,sep,thu,92.8,119,783.5,7.5,16.8,28,4,0,1.01+3,5,sep,thu,90.7,136.9,822.8,6.8,12.9,39,2.7,0,2.18+6,5,sep,thu,88.1,53.3,726.9,5.4,13.7,56,1.8,0,4.42+1,4,sep,sat,92.2,102.3,751.5,8.4,24.2,27,3.1,0,0+5,4,sep,sat,92.2,102.3,751.5,8.4,24.1,27,3.1,0,0+6,5,sep,sat,92.2,102.3,751.5,8.4,21.2,32,2.2,0,0+6,5,sep,sat,92.2,102.3,751.5,8.4,19.7,35,1.8,0,0+4,3,sep,sat,92.2,102.3,751.5,8.4,23.5,27,4,0,3.33+3,3,sep,sat,92.2,102.3,751.5,8.4,24.2,27,3.1,0,6.58+7,4,sep,sat,91.2,124.4,795.3,8.5,21.5,28,4.5,0,15.64+4,4,sep,sat,91.2,124.4,795.3,8.5,17.1,41,2.2,0,11.22+1,4,sep,mon,92.1,87.7,721.1,9.5,18.1,54,3.1,0,2.13+2,3,sep,mon,91.6,108.4,764,6.2,18,51,5.4,0,0+4,3,sep,mon,91.6,108.4,764,6.2,9.8,86,1.8,0,0+7,4,sep,mon,91.6,108.4,764,6.2,19.3,44,2.2,0,0+6,3,sep,mon,91.6,108.4,764,6.2,23,34,2.2,0,56.04+8,6,sep,mon,91.6,108.4,764,6.2,22.7,35,2.2,0,7.48+2,4,sep,mon,91.6,108.4,764,6.2,20.4,41,1.8,0,1.47+2,5,sep,mon,91.6,108.4,764,6.2,19.3,44,2.2,0,3.93+8,6,sep,mon,91.9,111.7,770.3,6.5,15.7,51,2.2,0,0+6,3,sep,mon,91.5,130.1,807.1,7.5,20.6,37,1.8,0,0+8,6,sep,mon,91.5,130.1,807.1,7.5,15.9,51,4.5,0,2.18+6,3,sep,mon,91.5,130.1,807.1,7.5,12.2,66,4.9,0,6.1+2,2,sep,mon,91.5,130.1,807.1,7.5,16.8,43,3.1,0,5.83+1,4,sep,mon,91.5,130.1,807.1,7.5,21.3,35,2.2,0,28.19+5,4,sep,fri,92.1,99,745.3,9.6,10.1,75,3.6,0,0+3,4,sep,fri,92.1,99,745.3,9.6,17.4,57,4.5,0,0+5,4,sep,fri,92.1,99,745.3,9.6,12.8,64,3.6,0,1.64+5,4,sep,fri,92.1,99,745.3,9.6,10.1,75,3.6,0,3.71+4,4,sep,fri,92.1,99,745.3,9.6,15.4,53,6.3,0,7.31+7,4,sep,fri,92.1,99,745.3,9.6,20.6,43,3.6,0,2.03+7,4,sep,fri,92.1,99,745.3,9.6,19.8,47,2.7,0,1.72+7,4,sep,fri,92.1,99,745.3,9.6,18.7,50,2.2,0,5.97+4,4,sep,fri,92.1,99,745.3,9.6,20.8,35,4.9,0,13.06+4,4,sep,fri,92.1,99,745.3,9.6,20.8,35,4.9,0,1.26+6,3,sep,fri,92.5,122,789.7,10.2,15.9,55,3.6,0,0+6,3,sep,fri,92.5,122,789.7,10.2,19.7,39,2.7,0,0+1,4,sep,fri,92.5,122,789.7,10.2,21.1,39,2.2,0,8.12+6,5,sep,fri,92.5,122,789.7,10.2,18.4,42,2.2,0,1.09+4,3,sep,fri,92.5,122,789.7,10.2,17.3,45,4,0,3.94+7,4,sep,fri,88.2,55.2,732.3,11.6,15.2,64,3.1,0,0.52+4,3,sep,tue,91.9,111.7,770.3,6.5,15.9,53,2.2,0,2.93+6,5,sep,tue,91.9,111.7,770.3,6.5,21.1,35,2.7,0,5.65+6,5,sep,tue,91.9,111.7,770.3,6.5,19.6,45,3.1,0,20.03+4,5,sep,tue,91.1,132.3,812.1,12.5,15.9,38,5.4,0,1.75+4,5,sep,tue,91.1,132.3,812.1,12.5,16.4,27,3.6,0,0+6,5,sep,sat,91.2,94.3,744.4,8.4,16.8,47,4.9,0,12.64+4,5,sep,sun,91,276.3,825.1,7.1,13.8,77,7.6,0,0+7,4,sep,sun,91,276.3,825.1,7.1,13.8,77,7.6,0,11.06+3,4,jul,wed,91.9,133.6,520.5,8,14.2,58,4,0,0+4,5,aug,sun,92,203.2,664.5,8.1,10.4,75,0.9,0,0+5,4,aug,thu,94.8,222.4,698.6,13.9,20.3,42,2.7,0,0+6,5,sep,fri,90.3,290,855.3,7.4,10.3,78,4,0,18.3+6,5,sep,sat,91.2,94.3,744.4,8.4,15.4,57,4.9,0,39.35+8,6,aug,mon,92.1,207,672.6,8.2,21.1,54,2.2,0,0+2,2,aug,sat,93.7,231.1,715.1,8.4,21.9,42,2.2,0,174.63+6,5,mar,thu,90.9,18.9,30.6,8,8.7,51,5.8,0,0+4,5,jan,sun,18.7,1.1,171.4,0,5.2,100,0.9,0,0+5,4,jul,wed,93.7,101.3,458.8,11.9,19.3,39,7.2,0,7.73+8,6,aug,thu,90.7,194.1,643,6.8,16.2,63,2.7,0,16.33+8,6,aug,wed,95.2,217.7,690,18,28.2,29,1.8,0,5.86+9,6,aug,thu,91.6,248.4,753.8,6.3,20.5,58,2.7,0,42.87+8,4,aug,sat,91.6,273.8,819.1,7.7,21.3,44,4.5,0,12.18+2,4,aug,sun,91.6,181.3,613,7.6,20.9,50,2.2,0,16+3,4,sep,sun,90.5,96.7,750.5,11.4,20.6,55,5.4,0,24.59+5,5,mar,thu,90.9,18.9,30.6,8,11.6,48,5.4,0,0+6,4,aug,fri,94.8,227,706.7,12,23.3,34,3.1,0,28.74+7,4,aug,fri,94.8,227,706.7,12,23.3,34,3.1,0,0+7,4,feb,mon,84.7,9.5,58.3,4.1,7.5,71,6.3,0,9.96+8,6,sep,fri,91.1,91.3,738.1,7.2,20.7,46,2.7,0,30.18+1,3,sep,sun,91,276.3,825.1,7.1,21.9,43,4,0,70.76+2,4,mar,tue,93.4,15,25.6,11.4,15.2,19,7.6,0,0+6,5,feb,mon,84.1,4.6,46.7,2.2,5.3,68,1.8,0,0+4,5,feb,sun,85,9,56.9,3.5,10.1,62,1.8,0,51.78+4,3,sep,sun,90.5,96.7,750.5,11.4,20.4,55,4.9,0,3.64+5,6,aug,sun,91.6,181.3,613,7.6,24.3,33,3.6,0,3.63+1,2,aug,sat,93.7,231.1,715.1,8.4,25.9,32,3.1,0,0+9,5,jun,wed,93.3,49.5,297.7,14,28,34,4.5,0,0+9,5,jun,wed,93.3,49.5,297.7,14,28,34,4.5,0,8.16+3,4,sep,thu,91.1,88.2,731.7,8.3,22.8,46,4,0,4.95+9,9,aug,fri,94.8,227,706.7,12,25,36,4,0,0+8,6,aug,thu,90.7,194.1,643,6.8,21.3,41,3.6,0,0+2,4,sep,wed,87.9,84.8,725.1,3.7,21.8,34,2.2,0,6.04+2,2,aug,tue,94.6,212.1,680.9,9.5,27.9,27,2.2,0,0+6,5,sep,sat,87.1,291.3,860.6,4,17,67,4.9,0,3.95+4,5,feb,sat,84.7,8.2,55,2.9,14.2,46,4,0,0+4,3,sep,fri,90.3,290,855.3,7.4,19.9,44,3.1,0,7.8+1,4,jul,tue,92.3,96.2,450.2,12.1,23.4,31,5.4,0,0+6,3,feb,fri,84.1,7.3,52.8,2.7,14.7,42,2.7,0,0+7,4,feb,fri,84.6,3.2,43.6,3.3,8.2,53,9.4,0,4.62+9,4,jul,mon,92.3,92.1,442.1,9.8,22.8,27,4.5,0,1.63+7,5,aug,sat,93.7,231.1,715.1,8.4,26.4,33,3.6,0,0+5,4,aug,sun,93.6,235.1,723.1,10.1,24.1,50,4,0,0+8,6,aug,thu,94.8,222.4,698.6,13.9,27.5,27,4.9,0,746.28+6,3,jul,tue,92.7,164.1,575.8,8.9,26.3,39,3.1,0,7.02+6,5,mar,wed,93.4,17.3,28.3,9.9,13.8,24,5.8,0,0+2,4,aug,sun,92,203.2,664.5,8.1,24.9,42,5.4,0,2.44+2,5,aug,sun,91.6,181.3,613,7.6,24.8,36,4,0,3.05+8,8,aug,wed,91.7,191.4,635.9,7.8,26.2,36,4.5,0,185.76+2,4,aug,wed,95.2,217.7,690,18,30.8,19,4.5,0,0+8,6,jul,sun,88.9,263.1,795.9,5.2,29.3,27,3.6,0,6.3+1,3,sep,sat,91.2,94.3,744.4,8.4,22.3,48,4,0,0.72+8,6,aug,sat,93.7,231.1,715.1,8.4,26.9,31,3.6,0,4.96+2,2,aug,thu,91.6,248.4,753.8,6.3,20.4,56,2.2,0,0+8,6,aug,thu,91.6,248.4,753.8,6.3,20.4,56,2.2,0,0+2,4,aug,mon,92.1,207,672.6,8.2,27.9,33,2.2,0,2.35+1,3,aug,thu,94.8,222.4,698.6,13.9,26.2,34,5.8,0,0+3,4,aug,sun,91.6,181.3,613,7.6,24.6,44,4,0,3.2+7,4,sep,thu,89.7,287.2,849.3,6.8,19.4,45,3.6,0,0+1,3,aug,sat,92.1,178,605.3,9.6,23.3,40,4,0,6.36+8,6,aug,thu,94.8,222.4,698.6,13.9,23.9,38,6.7,0,0+2,4,aug,sun,93.6,235.1,723.1,10.1,20.9,66,4.9,0,15.34+1,4,aug,fri,90.6,269.8,811.2,5.5,22.2,45,3.6,0,0+2,5,jul,sat,90.8,84.7,376.6,5.6,23.8,51,1.8,0,0+8,6,aug,mon,92.1,207,672.6,8.2,26.8,35,1.3,0,0.54+8,6,aug,sat,89.4,253.6,768.4,9.7,14.2,73,2.7,0,0+2,5,aug,sat,93.7,231.1,715.1,8.4,23.6,53,4,0,6.43+1,3,sep,fri,91.1,91.3,738.1,7.2,19.1,46,2.2,0,0.33+5,4,sep,fri,90.3,290,855.3,7.4,16.2,58,3.6,0,0+8,6,aug,mon,92.1,207,672.6,8.2,25.5,29,1.8,0,1.23+6,5,apr,mon,87.9,24.9,41.6,3.7,10.9,64,3.1,0,3.35+1,2,jul,fri,90.7,80.9,368.3,16.8,14.8,78,8,0,0+2,5,sep,fri,90.3,290,855.3,7.4,16.2,58,3.6,0,9.96+5,5,aug,sun,94,47.9,100.7,10.7,17.3,80,4.5,0,0+6,5,aug,sun,92,203.2,664.5,8.1,19.1,70,2.2,0,0+3,4,mar,wed,93.4,17.3,28.3,9.9,8.9,35,8,0,0+7,4,sep,wed,89.7,284.9,844,10.1,10.5,77,4,0,0+7,4,aug,sun,91.6,181.3,613,7.6,19.3,61,4.9,0,0+4,5,aug,wed,95.2,217.7,690,18,23.4,49,5.4,0,6.43+1,4,aug,fri,90.5,196.8,649.9,16.3,11.8,88,4.9,0,9.71+7,4,aug,mon,91.5,238.2,730.6,7.5,17.7,65,4,0,0+4,5,aug,thu,89.4,266.2,803.3,5.6,17.4,54,3.1,0,0+3,4,aug,thu,91.6,248.4,753.8,6.3,16.8,56,3.1,0,0+3,4,jul,mon,94.6,160,567.2,16.7,17.9,48,2.7,0,0+2,4,aug,thu,91.6,248.4,753.8,6.3,16.6,59,2.7,0,0+1,4,aug,wed,91.7,191.4,635.9,7.8,19.9,50,4,0,82.75+8,6,aug,sat,93.7,231.1,715.1,8.4,18.9,64,4.9,0,3.32+7,4,aug,sat,91.6,273.8,819.1,7.7,15.5,72,8,0,1.94+2,5,aug,sat,93.7,231.1,715.1,8.4,18.9,64,4.9,0,0+8,6,aug,sat,93.7,231.1,715.1,8.4,18.9,64,4.9,0,0+1,4,sep,sun,91,276.3,825.1,7.1,14.5,76,7.6,0,3.71+6,5,feb,tue,75.1,4.4,16.2,1.9,4.6,82,6.3,0,5.39+6,4,feb,tue,75.1,4.4,16.2,1.9,5.1,77,5.4,0,2.14+2,2,feb,sat,79.5,3.6,15.3,1.8,4.6,59,0.9,0,6.84+6,5,mar,mon,87.2,15.1,36.9,7.1,10.2,45,5.8,0,3.18+3,4,mar,wed,90.2,18.5,41.1,7.3,11.2,41,5.4,0,5.55+6,5,mar,thu,91.3,20.6,43.5,8.5,13.3,27,3.6,0,6.61+6,3,apr,sun,91,14.6,25.6,12.3,13.7,33,9.4,0,61.13+5,4,apr,sun,91,14.6,25.6,12.3,17.6,27,5.8,0,0+4,3,may,fri,89.6,25.4,73.7,5.7,18,40,4,0,38.48+8,3,jun,mon,88.2,96.2,229,4.7,14.3,79,4,0,1.94+9,4,jun,sat,90.5,61.1,252.6,9.4,24.5,50,3.1,0,70.32+4,3,jun,thu,93,103.8,316.7,10.8,26.4,35,2.7,0,10.08+2,5,jun,thu,93.7,121.7,350.2,18,22.7,40,9.4,0,3.19+4,3,jul,thu,93.5,85.3,395,9.9,27.2,28,1.3,0,1.76+4,3,jul,sun,93.7,101.3,423.4,14.7,26.1,45,4,0,7.36+7,4,jul,sun,93.7,101.3,423.4,14.7,18.2,82,4.5,0,2.21+7,4,jul,mon,89.2,103.9,431.6,6.4,22.6,57,4.9,0,278.53+9,9,jul,thu,93.2,114.4,560,9.5,30.2,25,4.5,0,2.75+4,3,jul,thu,93.2,114.4,560,9.5,30.2,22,4.9,0,0+3,4,aug,sun,94.9,130.3,587.1,14.1,23.4,40,5.8,0,1.29+8,6,aug,sun,94.9,130.3,587.1,14.1,31,27,5.4,0,0+2,5,aug,sun,94.9,130.3,587.1,14.1,33.1,25,4,0,26.43+2,4,aug,mon,95,135.5,596.3,21.3,30.6,28,3.6,0,2.07+5,4,aug,tue,95.1,141.3,605.8,17.7,24.1,43,6.3,0,2+5,4,aug,tue,95.1,141.3,605.8,17.7,26.4,34,3.6,0,16.4+4,4,aug,tue,95.1,141.3,605.8,17.7,19.4,71,7.6,0,46.7+4,4,aug,wed,95.1,141.3,605.8,17.7,20.6,58,1.3,0,0+4,4,aug,wed,95.1,141.3,605.8,17.7,28.7,33,4,0,0+4,4,aug,thu,95.8,152,624.1,13.8,32.4,21,4.5,0,0+1,3,aug,fri,95.9,158,633.6,11.3,32.4,27,2.2,0,0+1,3,aug,fri,95.9,158,633.6,11.3,27.5,29,4.5,0,43.32+6,6,aug,sat,96,164,643,14,30.8,30,4.9,0,8.59+6,6,aug,mon,96.2,175.5,661.8,16.8,23.9,42,2.2,0,0+4,5,aug,mon,96.2,175.5,661.8,16.8,32.6,26,3.1,0,2.77+3,4,aug,tue,96.1,181.1,671.2,14.3,32.3,27,2.2,0,14.68+6,5,aug,tue,96.1,181.1,671.2,14.3,33.3,26,2.7,0,40.54+7,5,aug,tue,96.1,181.1,671.2,14.3,27.3,63,4.9,6.4,10.82+8,6,aug,tue,96.1,181.1,671.2,14.3,21.6,65,4.9,0.8,0+7,5,aug,tue,96.1,181.1,671.2,14.3,21.6,65,4.9,0.8,0+4,4,aug,tue,96.1,181.1,671.2,14.3,20.7,69,4.9,0.4,0+2,4,aug,wed,94.5,139.4,689.1,20,29.2,30,4.9,0,1.95+4,3,aug,wed,94.5,139.4,689.1,20,28.9,29,4.9,0,49.59+1,2,aug,thu,91,163.2,744.4,10.1,26.7,35,1.8,0,5.8+1,2,aug,fri,91,166.9,752.6,7.1,18.5,73,8.5,0,0+2,4,aug,fri,91,166.9,752.6,7.1,25.9,41,3.6,0,0+1,2,aug,fri,91,166.9,752.6,7.1,25.9,41,3.6,0,0+5,4,aug,fri,91,166.9,752.6,7.1,21.1,71,7.6,1.4,2.17+6,5,aug,fri,91,166.9,752.6,7.1,18.2,62,5.4,0,0.43+8,6,aug,sun,81.6,56.7,665.6,1.9,27.8,35,2.7,0,0+4,3,aug,sun,81.6,56.7,665.6,1.9,27.8,32,2.7,0,6.44+2,4,aug,sun,81.6,56.7,665.6,1.9,21.9,71,5.8,0,54.29+7,4,aug,sun,81.6,56.7,665.6,1.9,21.2,70,6.7,0,11.16+1,4,aug,sat,94.4,146,614.7,11.3,25.6,42,4,0,0+6,3,nov,tue,79.5,3,106.7,1.1,11.8,31,4.5,0,0
+ examples/DemoPaths.hs view
@@ -0,0 +1,14 @@+module DemoPaths where++import qualified Frames.TH as Frames+import qualified Paths_Frames_streamly as Paths++forestFiresPath :: FilePath+forestFiresPath = "forestfires.csv"++thPath :: FilePath -> FilePath+thPath x = "./example_data/" ++ x+++usePath :: FilePath -> IO FilePath+usePath x = fmap (\dd -> dd ++ "/" ++ x) Paths.getDataDir
+ examples/Main.hs view
@@ -0,0 +1,40 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE TypeApplications #-}+{-# LANGUAGE TemplateHaskell #-}+module Main where++import qualified DemoPaths as Paths+import qualified Frames+import qualified Frames.Streamly.CSV as FStreamly+import qualified Frames.Streamly.InCore as FStreamly+import qualified Frames.Streamly.Transform as FStreamly+import qualified Data.Vinyl as V+import qualified Text.Printf as Printf+import qualified Streamly.Prelude as Streamly+import Data.Text (Text) -- for Frames template splicing +import qualified Data.Text as Text++Frames.tableTypes "ForestFires" (Paths.thPath Paths.forestFiresPath)++main :: IO ()+main = do+ forestFiresPath <- Paths.usePath Paths.forestFiresPath + forestFires :: Frames.Frame ForestFires <- FStreamly.inCoreAoS $ FStreamly.readTable forestFiresPath+ let filterAndMap :: ForestFires -> Maybe (Frames.Record [X, Y, Month, Day, Temp, Wind])+ filterAndMap r = if Frames.rgetField @Day r == "fri" then (Just $ Frames.rcast r) else Nothing+ forestFires' = FStreamly.mapMaybe filterAndMap forestFires+ formatRow = FStreamly.formatWithShow+ V.:& FStreamly.formatWithShow+ V.:& FStreamly.formatTextAsIs+ V.:& FStreamly.formatTextAsIs+ V.:& FStreamly.liftFieldFormatter (Text.pack . Printf.printf "%.1f")+ V.:& FStreamly.liftFieldFormatter (Text.pack . Printf.printf "%.1f")+ V.:& V.RNil+ csvTextStream = FStreamly.streamSV' formatRow "," $ Streamly.fromFoldable forestFires'+ Streamly.toList csvTextStream >>= putStrLn . Text.unpack . Text.intercalate "\n"+ FStreamly.writeLines "exampleOut.csv" csvTextStream+ +
+ src/Frames/Streamly/CSV.hs view
@@ -0,0 +1,599 @@+{-# LANGUAGE AllowAmbiguousTypes #-}+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE ConstraintKinds #-}+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DerivingStrategies #-}+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE FlexibleInstances #-}+{-# LANGUAGE KindSignatures #-}+{-# LANGUAGE MultiParamTypeClasses #-}+{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE RankNTypes #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE StrictData #-}+{-# LANGUAGE TupleSections #-}+{-# LANGUAGE TypeApplications #-}+{-# LANGUAGE TypeFamilies #-}+{-# LANGUAGE TypeOperators #-}+{-# LANGUAGE UndecidableInstances #-}+{-# LANGUAGE UndecidableSuperClasses #-}+{-|+Module : Frames.Streamly.CSV+Description : CSV parsing/formatting tools for the Frames library, operating via streamly Streams.+Copyright : (c) Adam Conner-Sax 2020+License : BSD-3-Clause+Maintainer : adam_conner_sax@yahoo.com+Stability : experimental++This module can be used in-place of Frames.CSV in order to use streamly streams where Frames uses pipes.+This module adds some functionality for formatting in more flexible ways than the pipes version in Frames.+It allows us of Show instances, in addition to the ShowCSV class included in Frames. And it allows one-off+specification of a format as well. See the example for more details.+-}+module Frames.Streamly.CSV+ (+ -- * read from File to Stream of Recs + readTable+ , readTableOpt+ , readTableMaybe+ , readTableMaybeOpt+ , readTableEither+ , readTableEitherOpt+ -- * convert streaming Text to streaming Records+ , streamTable+ , streamTableOpt+ , streamTableMaybe + , streamTableMaybeOpt+ , streamTableEither+ , streamTableEitherOpt+ -- * Produce (streaming) Text from Records+ , streamToCSV+ , streamCSV+ , streamToSV+ , streamSV+ , streamSV'+ -- * write Records to Text File+ , writeCSV+ , writeSV+ , writeStreamSV+ , writeCSV_Show+ , writeSV_Show+ , writeStreamSV_Show+ -- * Utilities+ , streamToList+ , liftFieldFormatter+ , liftFieldFormatter1+ , formatTextAsIs+ , formatWithShow+ , formatWithShowCSV+ , writeLines+ , writeLines'+ , word8ToTextLines+ )+where++import qualified Streamly.Prelude as Streamly+import qualified Streamly as Streamly+import Streamly ( IsStream )+import qualified Streamly.Data.Fold as Streamly.Fold+import qualified Streamly.Data.Unicode.Stream as Streamly.Unicode+import qualified Streamly.Internal.FileSystem.File as Streamly.File+import qualified Streamly.Internal.Data.Unfold as Streamly.Unfold+import Control.Monad.Catch ( MonadCatch )+import Control.Monad.IO.Class ( MonadIO )++import Data.Maybe (isNothing)+import qualified Data.Text as T++import qualified Data.Vinyl as Vinyl+import qualified Data.Vinyl.Functor as Vinyl+import qualified Data.Vinyl.TypeLevel as Vinyl+import qualified Data.Vinyl.Class.Method as Vinyl+import Data.Word ( Word8 )++import qualified Frames as Frames+import qualified Frames.CSV as Frames+import qualified Frames.ShowCSV as Frames ++import Data.Proxy (Proxy (..))++ ++-- | Given a stream of @Records@, for which all fields satisfy the `ShowCSV` constraint,+-- produce a stream of `Text`, one item (line) per `Record` with the specified separator+-- between fields.+streamToSV+ :: forall rs m t.+ ( Frames.ColumnHeaders rs+ , Monad m+ , Vinyl.RecordToList rs+ , Vinyl.RecMapMethod Frames.ShowCSV Vinyl.ElField rs+ , IsStream t+ )+ => T.Text -- ^ column separator+ -> t m (Frames.Record rs) -- ^ stream of Records+ -> t m T.Text -- ^ stream of 'Text' rows+streamToSV = streamSVClass @Frames.ShowCSV Frames.showCSV+{-# INLINEABLE streamToSV #-}++-- | Given a stream of @Records@, for which all fields satisfy the `ShowCSV` constraint,+-- produce a stream of CSV `Text`, one item (line) per `Record`.+streamToCSV+ :: forall rs m t+ . ( Frames.ColumnHeaders rs+ , Monad m+ , Vinyl.RecordToList rs+ , Vinyl.RecMapMethod Frames.ShowCSV Vinyl.ElField rs+ , IsStream t+ )+ => t m (Frames.Record rs) -- ^ stream of Records+ -> t m T.Text -- ^ stream of 'Text' rows+streamToCSV = streamToSV ","+{-# INLINEABLE streamToCSV #-}++-- | Given a foldable of @Records@, for which all fields satisfy the `ShowCSV` constraint,+-- produce a stream of `Text`, one item (line) per `Record` with the specified separator+-- between fields. +streamSV+ :: forall f rs m t.+ ( Frames.ColumnHeaders rs+ , Foldable f+ , Monad m+ , Vinyl.RecordToList rs+ , Vinyl.RecMapMethod Frames.ShowCSV Vinyl.ElField rs+ , IsStream t+ )+ => T.Text -- ^ column separator+ -> f (Frames.Record rs) -- ^ foldable of Records+ -> t m T.Text -- ^ stream of 'Text' rows+streamSV sep = streamToSV sep . Streamly.fromFoldable +{-# INLINEABLE streamSV #-}++-- | Given a foldable of @Records@, for which all fields satisfy the `ShowCSV` constraint,+-- produce a stream of CSV `Text`, one item (line) per `Record`.+streamCSV+ :: forall f rs m t.+ ( Frames.ColumnHeaders rs+ , Foldable f+ , Monad m+ , Vinyl.RecordToList rs+ , Vinyl.RecMapMethod Frames.ShowCSV Vinyl.ElField rs+ , IsStream t+ )+ => f (Frames.Record rs) -- ^ 'Foldable' of Records+ -> t m T.Text -- ^ stream of 'Text' rows+streamCSV = streamSV ","++-- | Convert @Rec@s to lines of `Text` using a class (which must have an instance+-- for each type in the record) to covert each field to `Text`.+streamSVClass+ :: forall c rs t m .+ ( Vinyl.RecMapMethod c Vinyl.ElField rs+ , Vinyl.RecordToList rs+ , Frames.ColumnHeaders rs+ , IsStream t+ , Monad m+ )+ => (forall a. c a => a -> T.Text) -- ^ @show@-like function for some constraint satisfied by all fields.+ -> T.Text -- ^ column separator+ -> t m (Frames.Record rs) -- ^ stream of Records+ -> t m T.Text -- ^ stream of 'Text' rows+streamSVClass toText sep s =+ (T.intercalate sep . fmap T.pack $ Frames.columnHeaders (Proxy :: Proxy (Frames.Record rs)))+ `Streamly.cons`+ (Streamly.map (T.intercalate sep . Vinyl.recordToList . Vinyl.rmapMethod @c aux) s)+ where+ aux :: (c (Vinyl.PayloadType Vinyl.ElField a))+ => Vinyl.ElField a+ -> Vinyl.Const T.Text a+ aux (Vinyl.Field x) = Vinyl.Const $ toText x++ +-- | Given a record of functions to map each field to Text,+-- transform a stream of records into a stream of lines of Text,+-- headers first, with headers/fields separated by the given separator.+streamSV'+ :: forall rs t m f.+ (Vinyl.RecordToList rs+ , Vinyl.RApply rs+ , Frames.ColumnHeaders rs+ , IsStream t+ , Monad m+ )+ => Vinyl.Rec (Vinyl.Lift (->) f (Vinyl.Const T.Text)) rs -- ^ Vinyl record of formatting functions for the row-type.+ -> T.Text -- ^ column separator+ -> t m (Frames.Rec f rs) -- ^ stream of Records+ -> t m T.Text -- ^ stream of 'Text' rows+streamSV' toTextRec sep s = + (T.intercalate sep . fmap T.pack $ Frames.columnHeaders (Proxy :: Proxy (Frames.Record rs)))+ `Streamly.cons`+ (Streamly.map (T.intercalate sep . Vinyl.recordToList . Vinyl.rapply toTextRec) s)+{-# INLINEABLE streamSV' #-}++-- | Convert a streamly stream into a (lazy) list+streamToList :: (IsStream t, Monad m) => t m a -> m [a] +streamToList = Streamly.toList . Streamly.adapt++-- | lift a field formatting function into the right form to append to a Rec of formatters+liftFieldFormatter :: Vinyl.KnownField t+ => (Vinyl.Snd t -> T.Text) -- ^ formatting function for the type in Field @t@+ -> Vinyl.Lift (->) Vinyl.ElField (Vinyl.Const T.Text) t -- ^ formatting function in the form required to use in row-formatters.+liftFieldFormatter toText = Vinyl.Lift $ Vinyl.Const . toText . Vinyl.getField+{-# INLINEABLE liftFieldFormatter #-}++-- | lift a composed-field formatting function into the right form to append to a Rec of formatters+-- | Perhaps to format a parsed file with @Maybe@ or @Either@ composed with @ElField@+liftFieldFormatter1 :: (Functor f, Vinyl.KnownField t)+ => (f (Vinyl.Snd t) -> T.Text) -- ^ formatting function for things like @Maybe a@+ -> Vinyl.Lift (->) (f Vinyl.:. Vinyl.ElField) (Vinyl.Const T.Text) t+liftFieldFormatter1 toText = Vinyl.Lift $ Vinyl.Const . toText . fmap Vinyl.getField . Vinyl.getCompose+{-# INLINEABLE liftFieldFormatter1 #-}++-- | Format a @Text@ field as-is.+formatTextAsIs :: (Vinyl.KnownField t, Vinyl.Snd t ~ T.Text) => Vinyl.Lift (->) Vinyl.ElField (Vinyl.Const T.Text) t+formatTextAsIs = liftFieldFormatter id+{-# INLINE formatTextAsIs #-}++-- | Format a field using the @Show@ instance of the contained type +formatWithShow :: (Vinyl.KnownField t, Show (Vinyl.Snd t)) => Vinyl.Lift (->) Vinyl.ElField (Vinyl.Const T.Text) t+formatWithShow = liftFieldFormatter $ T.pack . show+{-# INLINE formatWithShow #-}++-- | Format a field using the @Frames.ShowCSV@ instance of the contained type +formatWithShowCSV :: (Vinyl.KnownField t, Frames.ShowCSV (Vinyl.Snd t)) => Vinyl.Lift (->) Vinyl.ElField (Vinyl.Const T.Text) t+formatWithShowCSV = liftFieldFormatter Frames.showCSV+{-# INLINE formatWithShowCSV #-}++-- NB: Uses some internal modules from Streamly. Will have to change when they become stable+-- | write a stream of @Text@ to a file, one line per stream item.+writeLines' :: (Streamly.MonadAsync m, MonadCatch m, Streamly.IsStream t) => FilePath -> t m T.Text -> m ()+writeLines' fp s = do+ Streamly.fold (Streamly.File.write fp)+ $ Streamly.Unicode.encodeUtf8+ $ Streamly.adapt+ $ Streamly.concatUnfold Streamly.Unfold.fromList+ $ Streamly.map T.unpack+ $ Streamly.intersperse "\n" s+{-# INLINEABLE writeLines' #-}++-- | write a stream of @Text@ to a file, one line per stream item.+-- | Monomorphised to serial streams for ease of use.+writeLines :: (Streamly.MonadAsync m, MonadCatch m) => FilePath -> Streamly.SerialT m T.Text -> m ()+writeLines = writeLines'+{-# INLINE writeLines #-}++-- NB: Uses some internal modules from Streamly. Will have to change when they become stable+-- | write a stream of @Records@ to a file, one line per @Record@.+-- Use the 'Frames.ShowCSV' class to format each field to @Text@+writeStreamSV+ :: forall rs m t.+ ( Frames.ColumnHeaders rs+ , MonadCatch m+ , Vinyl.RecordToList rs+ , Vinyl.RecMapMethod Frames.ShowCSV Vinyl.ElField rs+ , IsStream t+ , Streamly.MonadAsync m+ )+ => T.Text -- ^ column separator+ -> FilePath -- ^ path+ -> t m (Frames.Record rs) -- ^ stream of Records+ -> m ()+writeStreamSV sep fp = writeLines' fp . streamToSV sep +{-# INLINEABLE writeStreamSV #-}++-- | write a foldable of @Records@ to a file, one line per @Record@.+-- Use the 'Frames.ShowCSV' class to format each field to @Text@+writeSV+ :: forall rs m f.+ ( Frames.ColumnHeaders rs+ , MonadCatch m+ , Vinyl.RecordToList rs+ , Vinyl.RecMapMethod Frames.ShowCSV Vinyl.ElField rs+ , Streamly.MonadAsync m+ , Foldable f+ )+ => T.Text -- ^ column separator+ -> FilePath -- ^ file path+ -> f (Frames.Record rs) -- ^ Foldable of Records+ -> m ()+writeSV sep fp = writeStreamSV sep fp . Streamly.fromFoldable @Streamly.AheadT+{-# INLINEABLE writeSV #-}++-- | write a foldable of @Records@ to a file, one line per @Record@.+-- Use the 'Frames.ShowCSV' class to format each field to @Text@+writeCSV+ :: forall rs m f.+ ( Frames.ColumnHeaders rs+ , MonadCatch m+ , Vinyl.RecordToList rs+ , Vinyl.RecMapMethod Frames.ShowCSV Vinyl.ElField rs+ , Streamly.MonadAsync m+ , Foldable f+ )+ => FilePath -- ^ file path+ -> f (Frames.Record rs) -- ^ 'Foldable' of Records+ -> m ()+writeCSV fp = writeSV "," fp +{-# INLINEABLE writeCSV #-}++-- NB: Uses some internal modules from Streamly. Will have to change when they become stable+-- | write a stream of @Records@ to a file, one line per @Record@.+-- Use the 'Show' class to format each field to @Text@+writeStreamSV_Show+ :: forall rs m t.+ ( Frames.ColumnHeaders rs+ , MonadCatch m+ , Vinyl.RecordToList rs+ , Vinyl.RecMapMethod Show Vinyl.ElField rs+ , IsStream t+ , Streamly.MonadAsync m+ )+ => T.Text -- ^ column separator+ -> FilePath -- ^ file path+ -> t m (Frames.Record rs) -- ^ stream of Records+ -> m ()+writeStreamSV_Show sep fp = writeLines' fp . streamSVClass @Show (T.pack . show) sep+{-# INLINEABLE writeStreamSV_Show #-}++-- | write a foldable of @Records@ to a file, one line per @Record@.+-- Use the 'Show' class to format each field to @Text@+writeSV_Show+ :: forall rs m f.+ ( Frames.ColumnHeaders rs+ , MonadCatch m+ , Vinyl.RecordToList rs+ , Vinyl.RecMapMethod Show Vinyl.ElField rs+ , Streamly.MonadAsync m+ , Foldable f+ )+ => T.Text -- ^ column separator+ -> FilePath -- ^ file path+ -> f (Frames.Record rs) -- ^ 'Foldable' of Records+ -> m ()+writeSV_Show sep fp = writeStreamSV_Show sep fp . Streamly.fromFoldable @Streamly.AheadT+{-# INLINEABLE writeSV_Show #-}++-- | write a foldable of @Records@ to a file, one line per @Record@.+-- Use the 'Show' class to format each field to @Text@+writeCSV_Show+ :: forall rs m f.+ ( Frames.ColumnHeaders rs+ , MonadCatch m+ , Vinyl.RecordToList rs+ , Vinyl.RecMapMethod Show Vinyl.ElField rs+ , Streamly.MonadAsync m+ , Foldable f+ )+ => FilePath -- ^ file path+ -> f (Frames.Record rs) -- ^ 'Foldable' of Records+ -> m ()+writeCSV_Show fp = writeSV_Show "," fp +{-# INLINEABLE writeCSV_Show #-}++-- Thanks to Tim Pierson for the functions below!++-- | Stream a table from a file path, using the default options.+-- Results composed with the @Maybe@ functor. Unparsed fields are returned as @Nothing@.+-- NB: If the inferred/given rs is different from the actual file row-type, things will go awry.+readTableMaybe+ :: forall rs t m.+ (MonadIO m+ , IsStream t+ , Vinyl.RMap rs+ , Frames.ReadRec rs+ , MonadCatch m)+ => FilePath -- ^ file path+ -> t m (Vinyl.Rec (Maybe Vinyl.:. Vinyl.ElField) rs) -- ^ stream of @Maybe :. ElField@ records after parsing. +readTableMaybe = readTableMaybeOpt Frames.defaultParser+{-# INLINEABLE readTableMaybe #-}++-- | Stream a table from a file path.+-- Results composed with the @Maybe@ functor. Unparsed fields are returned as @Nothing@.+-- NB: If the inferred/given rs is different from the actual file row-type, things will go awry.+readTableMaybeOpt+ :: forall rs t m.+ (MonadIO m+ , IsStream t+ , Vinyl.RMap rs+ , Frames.ReadRec rs+ , MonadCatch m)+ => Frames.ParserOptions -- ^ parsing options+ -> FilePath -- ^ file path+ -> t m (Vinyl.Rec (Maybe Vinyl.:. Vinyl.ElField) rs) -- ^ stream of @Maybe :. ElField@ records after parsing. +readTableMaybeOpt opts = Streamly.map recEitherToMaybe . readTableEitherOpt opts+{-# INLINEABLE readTableMaybeOpt #-}++-- | Stream a table from a file path.+-- Results composed with the @Either Text@ functor. Unparsed fields are returned as a @Left@+-- containing the string that failed to parse.+-- Uses default options.+-- NB: If the inferred/given rs is different from the actual file row-type, things will go awry.+readTableEither+ :: forall rs t m.+ (MonadIO m+ , IsStream t+ , Vinyl.RMap rs+ , Frames.ReadRec rs+ , MonadCatch m)+ => FilePath -- ^ file path+ -> t m (Vinyl.Rec (Either T.Text Vinyl.:. Vinyl.ElField) rs) -- ^ stream of @Either :. ElField@ records after parsing. +readTableEither = readTableEitherOpt Frames.defaultParser++-- | Stream a table from a file path.+-- Results composed with the @Either Text@ functor. Unparsed fields are returned as a @Left@+-- containing the string that failed to parse.+-- NB: If the inferred/given rs is different from the actual file row-type, things will go awry.+readTableEitherOpt+ :: forall rs t m.+ (MonadIO m+ , IsStream t+ , Vinyl.RMap rs+ , Frames.ReadRec rs+ , MonadCatch m)+ => Frames.ParserOptions -- ^ parsing options+ -> FilePath -- ^ file path+ -> t m (Vinyl.Rec (Either T.Text Vinyl.:. Vinyl.ElField) rs) -- ^ stream of @Either :. ElField@ records after parsing. +readTableEitherOpt opts = streamTableEitherOpt opts . word8ToTextLines . Streamly.File.toBytes +{-# INLINEABLE readTableEitherOpt #-}+++-- | Stream Table from a file path, dropping rows where any field fails to parse+-- | Use default options+-- NB: If the inferred/given @rs@ is different from the actual file row-type, things will go awry.+readTable+ :: forall rs t m.+ (MonadIO m+ , IsStream t+ , Vinyl.RMap rs+ , Frames.ReadRec rs+ , MonadCatch m)+ => FilePath -- ^ file path+ -> t m (Frames.Record rs) -- ^ stream of Records+readTable = readTableOpt Frames.defaultParser+{-# INLINEABLE readTable #-}++-- | Stream Table from a file path, dropping rows where any field fails to parse+-- NB: If the inferred/given @rs@ is different from the actual file row-type, things will go awry.+readTableOpt+ :: forall rs t m.+ (MonadIO m+ , IsStream t+ , Vinyl.RMap rs+ , Frames.ReadRec rs+ , MonadCatch m)+ => Frames.ParserOptions -- ^ parsing options+ -> FilePath -- ^ file path+ -> t m (Frames.Record rs) -- ^ stream of Records+readTableOpt opts = streamTableOpt opts . word8ToTextLines . Streamly.File.toBytes +{-# INLINEABLE readTableOpt #-}++-- | Convert a stream of lines of `Text` to a table+-- Each field is returned in an @Either Text@ functor. @Right a@ for successful parses+-- and @Left Text@ when parsing fails, containing the text that failed to Parse.+--+-- NB: If the inferred/given @rs@ is different from the actual file row-type, things will go awry.+streamTableEither+ :: forall rs t m.+ (MonadIO m+ , IsStream t+ , Vinyl.RMap rs+ , Frames.ReadRec rs)+ => t m T.Text -- ^ stream of 'Text' rows+ -> t m (Vinyl.Rec ((Either T.Text) Vinyl.:. Vinyl.ElField) rs) -- ^ stream of parsed @Either :. ElField@ rows+streamTableEither = streamTableEitherOpt Frames.defaultParser+{-# INLINEABLE streamTableEither #-}++-- | Convert a stream of lines of `Text` to records.+-- Each field is returned in an @Either Text@ functor. @Right a@ for successful parses+-- and @Left Text@ when parsing fails, containing the text that failed to Parse.+--+-- NB: If the inferred/given @rs@ is different from the actual file row-type, things will..go awry.+streamTableEitherOpt+ :: forall rs t m.+ (MonadIO m+ , IsStream t+ , Vinyl.RMap rs+ , Frames.ReadRec rs)+ => Frames.ParserOptions -- ^ parsing options+ -> t m T.Text -- ^ stream of 'Text' rows+ -> t m (Vinyl.Rec ((Either T.Text) Vinyl.:. Vinyl.ElField) rs) -- ^ stream of parsed @Either :. ElField@ rows+streamTableEitherOpt opts =+ Streamly.map (doParse . Frames.tokenizeRow opts)+ . handleHeader+ where+ handleHeader | isNothing (Frames.headerOverride opts) = Streamly.drop 1+ | otherwise = id+ doParse = Frames.readRec +{-# INLINEABLE streamTableEitherOpt #-}++-- | Convert a stream of lines of `Text` to a table.+--+-- NB: If the inferred/given @rs@ is different from the actual file row-type, things will..go awry.+streamTableMaybe+ :: forall rs t m.+ (MonadIO m+ , IsStream t+ , Vinyl.RMap rs+ , Frames.ReadRec rs)+ => t m T.Text -- ^ stream of 'Text' rows + -> t m (Vinyl.Rec (Maybe Vinyl.:. Vinyl.ElField) rs) -- ^ stream of parsed @Maybe :. ElField@ rows+streamTableMaybe = streamTableMaybeOpt Frames.defaultParser +{-# INLINEABLE streamTableMaybe #-}++-- | Convert a stream of lines of Text to a table .+--+-- NB: If the inferred/given @rs@ is different from the actual file row-type, things will..go awry.+streamTableMaybeOpt+ :: forall rs t m.+ (MonadIO m+ , IsStream t+ , Vinyl.RMap rs+ , Frames.ReadRec rs)+ => Frames.ParserOptions -- ^ parsing options+ -> t m T.Text -- ^ stream of 'Text' rows+ -> t m (Vinyl.Rec (Maybe Vinyl.:. Vinyl.ElField) rs) -- ^ stream of parsed @Maybe :. ElField@ rows+streamTableMaybeOpt opts = Streamly.map recEitherToMaybe . streamTableEitherOpt opts+{-# INLINEABLE streamTableMaybeOpt #-}++-- | Convert a stream of lines of 'Text' to a table,+-- dropping rows where any field fails to parse.+-- Use default options.+-- NB: If the inferred/given @rs@ is different from the actual file row-type, things will go awry.+streamTable+ :: forall rs t m.+ (MonadIO m+ , IsStream t+ , Vinyl.RMap rs+ , Frames.ReadRec rs+ )+ => t m T.Text -- ^ stream of 'Text' rows+ -> t m (Frames.Record rs) -- ^ stream of Records+streamTable = streamTableOpt Frames.defaultParser+{-# INLINEABLE streamTable #-}++-- | Convert a stream of lines of 'Text' `Word8` to a table,+-- dropping rows where any field fails to parse.+-- NB: If the inferred/given @rs@ is different from the actual file row-type, things will go awry.+streamTableOpt+ :: forall rs t m.+ (MonadIO m+ , IsStream t+ , Vinyl.RMap rs+ , Frames.ReadRec rs+ )+ => Frames.ParserOptions -- ^ parsing options+ -> t m T.Text -- ^ stream of 'Text' rows+ -> t m (Frames.Record rs) -- ^ stream of Records+streamTableOpt opts =+ Streamly.mapMaybe (Frames.recMaybe . doParse . Frames.tokenizeRow opts)+ . handleHeader + where+ handleHeader | isNothing (Frames.headerOverride opts) = Streamly.drop 1+ | otherwise = id+ doParse = recEitherToMaybe . Frames.readRec+{-# INLINE streamTableOpt #-}++recEitherToMaybe :: Vinyl.RMap rs => Vinyl.Rec (Either T.Text Vinyl.:. Vinyl.ElField) rs -> Vinyl.Rec (Maybe Vinyl.:. Vinyl.ElField) rs+recEitherToMaybe = Vinyl.rmap (either (const (Vinyl.Compose Nothing)) (Vinyl.Compose . Just) . Vinyl.getCompose)+{-# INLINE recEitherToMaybe #-}++-- | Convert a stream of Word8 to lines of `Text` by decoding as UTF8 and splitting on "\n"+word8ToTextLines :: (IsStream t, Monad m) => t m Word8 -> t m T.Text+word8ToTextLines = Streamly.splitOnSuffix (== '\n') (fmap T.pack $ Streamly.Fold.toList)+ . Streamly.Unicode.decodeUtf8+{-# INLINE word8ToTextLines #-}+++-- tracing fold+{-+runningCountF :: MonadIO m => T.Text -> (Int -> T.Text) -> T.Text -> Streamly.Fold.Fold m a ()+runningCountF startMsg countMsg endMsg = Streamly.Fold.Fold step start done where+ start = liftIO (T.putStr startMsg) >> return 0+ step !n _ = liftIO $ do+ t <- System.Clock.getTime System.Clock.ProcessCPUTime+ putStr $ show t ++ ": "+ T.putStrLn $ countMsg n+ return (n+1)+ done _ = liftIO $ T.putStrLn endMsg+-}+
+ src/Frames/Streamly/InCore.hs view
@@ -0,0 +1,114 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DerivingStrategies #-}+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE RankNTypes #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE StrictData #-}+{-# LANGUAGE TupleSections #-}+{-# LANGUAGE TypeApplications #-}+{-# LANGUAGE TypeOperators #-}+{-|+Module : Frames.Streamly.InCore+Description : Support for transformations between streamly streams and Frames Array of Structures (SoA).+Copyright : (c) Adam Conner-Sax 2020+License : BSD-3-Clause+Maintainer : adam_conner_sax@yahoo.com+Stability : experimental++This module can be used in-place of Frames.InCore. The pipes functions are all replaced by equivalent streamly functions.+Relevant classes and type-families are re-exported for convenience.+-}+module Frames.Streamly.InCore+ (+ inCoreSoA+ , inCoreAoS+ , inCoreAoS'+ , inCoreSoA_F+ , inCoreAoS_F+ , inCoreAoS'_F+ -- * Re-exports from Frames+ , toAoS+ , VectorFor+ , VectorMFor+ , VectorMs+ , Vectors+ , RecVec(..) + + )+where++import qualified Streamly as Streamly+import qualified Streamly.Prelude as Streamly+import qualified Streamly.Data.Fold as Streamly.Fold+import qualified Streamly.Internal.Data.Fold as Streamly.Fold++import qualified Control.Monad.Primitive as Prim++import qualified Data.Vinyl as Vinyl++import qualified Frames as Frames+import qualified Frames.InCore as Frames+import Frames.InCore (VectorFor, VectorMFor, VectorMs, Vectors, RecVec(..), toAoS)++import Data.Proxy (Proxy(..))+++-- | Fold a stream of 'Vinyl' records into SoA (Structure-of-Arrays) form.+-- Here as a 'streamly' fold, so it may be deployed along with other folds or on only part of a stream.+inCoreSoA_F :: forall m rs. (Prim.PrimMonad m, Frames.RecVec rs)+ => Streamly.Fold.Fold m (Frames.Record rs) (Int, Vinyl.Rec (((->) Int) Frames.:. Frames.ElField) rs)+inCoreSoA_F = Streamly.Fold.mkFold feed initial fin+ where feed (!i, !sz, !mvs') row+ | i == sz = Frames.growRec (Proxy::Proxy rs) mvs'+ >>= flip feed row . (i, sz*2,)+ | otherwise = do Frames.writeRec (Proxy::Proxy rs) i mvs' row+ return (i+1, sz, mvs')+ + initial = do+ mvs <- Frames.allocRec (Proxy :: Proxy rs) Frames.initialCapacity+ return (0, Frames.initialCapacity, mvs)+ + fin (n, _, mvs') =+ do vs <- Frames.freezeRec (Proxy::Proxy rs) n mvs'+ return . (n,) $ Frames.produceRec (Proxy::Proxy rs) vs+{-# INLINE inCoreSoA_F #-}++-- | Perform the 'inCoreSoA_F' fold on a stream of records.+inCoreSoA :: forall m rs. (Prim.PrimMonad m, Frames.RecVec rs)+ => Streamly.SerialT m (Frames.Record rs)+ -> m (Int, Vinyl.Rec (((->) Int) Frames.:. Frames.ElField) rs)+inCoreSoA = Streamly.fold inCoreSoA_F+{-# INLINE inCoreSoA #-}++-- | Fold a stream of 'Vinyl' records into AoS (Array-of-Structures) form.+inCoreAoS_F :: forall m rs. (Prim.PrimMonad m, Frames.RecVec rs)+ => Streamly.Fold.Fold m (Frames.Record rs) (Frames.FrameRec rs)+inCoreAoS_F = fmap (uncurry Frames.toAoS) inCoreSoA_F+{-# INLINE inCoreAoS_F #-}++-- | Perform the 'inCoreAoS_F' fold on a stream of records.+inCoreAoS :: forall m rs. (Prim.PrimMonad m, Frames.RecVec rs)+ => Streamly.SerialT m (Frames.Record rs)+ -> m (Frames.FrameRec rs)+inCoreAoS = Streamly.fold inCoreAoS_F --fmap (uncurry Frames.toAoS) . inCoreSoA+{-# INLINE inCoreAoS #-}+++-- | More general AoS fold, allowing for a, possible column changing, transformation of the records while in SoA form.+inCoreAoS'_F :: forall ss rs m. (Prim.PrimMonad m, Frames.RecVec rs)+ => (Frames.Rec ((->) Int Frames.:. Frames.ElField) rs -> Frames.Rec ((->) Int Frames.:. Frames.ElField) ss)+ -> Streamly.Fold.Fold m (Frames.Record rs) (Frames.FrameRec ss)+inCoreAoS'_F f = fmap (uncurry Frames.toAoS . aux) inCoreSoA_F+ where aux (x,y) = (x, f y)+{-# INLINE inCoreAoS'_F #-} ++-- | Perform the more general AoS fold on a stream of records.+inCoreAoS' :: forall ss rs m. (Prim.PrimMonad m, Frames.RecVec rs)+ => (Frames.Rec ((->) Int Frames.:. Frames.ElField) rs -> Frames.Rec ((->) Int Frames.:. Frames.ElField) ss)+ -> Streamly.SerialT m (Frames.Record rs)+ -> m (Frames.FrameRec ss)+inCoreAoS' f = Streamly.fold (inCoreAoS'_F f)+{-# INLINE inCoreAoS' #-} +
+ src/Frames/Streamly/Transform.hs view
@@ -0,0 +1,93 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE DerivingStrategies #-}+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE RankNTypes #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE StrictData #-}+{-# LANGUAGE TupleSections #-}+{-# LANGUAGE TypeApplications #-}+{-# LANGUAGE TypeOperators #-}+{-|+Module : Frames.Streamly.Transform+Description : Support for transformations on Frames using streamly streams as an intermediate structure.+Copyright : (c) Adam Conner-Sax 2020+License : BSD-3-Clause+Maintainer : adam_conner_sax@yahoo.com+Stability : experimental++This module adds some functions for using streamly to make transformations on Frames that are either:++1. Like filtering or some one-to-many mapping of rows,+whic makes the efficient memory layout useless in-flight. Frames already implements 'filterFrame'+this way, using Pipes and the ST monad.++2. Expensive enough per row that the concurrency features of streams become worthwhile.+-}+module Frames.Streamly.Transform+ ( transform+ , filter+ , concurrentMapM+ , mapMaybe+ , concurrentMapMaybeM+ )+where++import qualified Frames.Streamly.InCore as FS+import Prelude hiding (filter)++import qualified Streamly as Streamly+import qualified Streamly.Prelude as Streamly+import Streamly ( IsStream )+import qualified Control.Monad.Primitive as Prim+import Control.Monad.ST (runST)+import qualified Frames as Frames+import qualified Frames.InCore as Frames+++++-- | Use streamly to transform a frame.+transform ::+ forall t as bs m.+ (IsStream t+ , Prim.PrimMonad m+ , Frames.RecVec as+ , Frames.RecVec bs+ )+ => (t m (Frames.Record as) -> Streamly.SerialT m (Frames.Record bs)) -> Frames.FrameRec as -> m (Frames.FrameRec bs)+transform f = FS.inCoreAoS . f . Streamly.fromFoldable+{-# INLINE transform #-}++-- | Filter using streamly +filter :: (Frames.RecVec as) => (Frames.Record as -> Bool) -> Frames.FrameRec as -> Frames.FrameRec as+filter f frame = runST $ transform (Streamly.serially . Streamly.filter f) frame+{-# INLINE filter #-}++-- | map using speculative streams (concurrency that preserves ordering of results).+concurrentMapM :: (Prim.PrimMonad m+ , Streamly.MonadAsync m + , Frames.RecVec as+ , Frames.RecVec bs+ ) => (Frames.Record as -> m (Frames.Record bs)) -> Frames.FrameRec as -> m (Frames.FrameRec bs)+concurrentMapM f = transform (Streamly.aheadly . Streamly.mapM f)+{-# INLINE concurrentMapM #-}++-- | mapMaybe using streamly+mapMaybe :: (Frames.RecVec as+ , Frames.RecVec bs+ ) => (Frames.Record as -> Maybe (Frames.Record bs)) -> Frames.FrameRec as -> Frames.FrameRec bs+mapMaybe f frame = runST $ transform (Streamly.aheadly . Streamly.mapMaybe f) frame+{-# INLINE mapMaybe #-}++-- | mapMaybeM using speculative streams (concurrency that preserves ordering of results).+concurrentMapMaybeM :: (Prim.PrimMonad m+ , Streamly.MonadAsync m + , Frames.RecVec as+ , Frames.RecVec bs+ ) => (Frames.Record as -> m (Maybe (Frames.Record bs))) -> Frames.FrameRec as -> m (Frames.FrameRec bs)+concurrentMapMaybeM f = transform (Streamly.aheadly . Streamly.mapMaybeM f)+{-# INLINE concurrentMapMaybeM #-}++