dataframe 0.3.3.1 → 0.3.3.2
raw patch · 5 files changed
+37/−106 lines, 5 filesPVP ok
version bump matches the API change (PVP)
API changes (from Hackage documentation)
Files
- CHANGELOG.md +3/−0
- README.md +1/−95
- dataframe.cabal +1/−1
- src/DataFrame.hs +30/−7
- src/DataFrame/Operations/Subset.hs +2/−3
CHANGELOG.md view
@@ -1,5 +1,8 @@ # Revision history for dataframe +## 0.3.3.2+* Update documentation on both readthedocs and hackage.+ ## 0.3.3.1 * Fix bug in `randomSplit` causing two splits to overlap.
README.md view
@@ -31,98 +31,4 @@ * Designed for interactivity: expressive syntax, helpful error messages, and sensible defaults. * Works seamlessly in both command-line and notebook environments—great for exploration and scripting alike. -## Example usage--### Interactive environment---Key features in example:-* Intuitive, SQL-like API to get from data to insights.-* Create typed, completion-ready references to columns in a dataframe using `:exposeColumns`-* Type-safe column transformations for faster and safer exploration.-* Fluid, chaining API that makes code easy to reason about.--### Standalone script example-```haskell--- Useful Haskell extensions.-{-# LANGUAGE OverloadedStrings #-} -- Allow string literal to be interpreted as any other string type.-{-# LANGUAGE TypeApplications #-} -- Convenience syntax for specifiying the type `sum a b :: Int` vs `sum @Int a b'. --import qualified DataFrame as D -- import for general functionality.-import qualified DataFrame.Functions as F -- import for column expressions.--import DataFrame ((|>)) -- import chaining operator with unqualified.--main :: IO ()-main = do- df <- D.readTsv "./data/chipotle.tsv"- let quantity = F.col "quantity" :: D.Expr Int -- A typed reference to a column.- print (df- |> D.select ["item_name", "quantity"]- |> D.groupBy ["item_name"]- |> D.aggregate [ (F.sum quantity) `F.as` "sum_quantity"- , (F.mean quantity) `F.as` "mean_quantity"- , (F.maximum quantity) `F.as` "maximum_quantity"- ]- |> D.sortBy D.Descending ["sum_quantity"]- |> D.take 10)--```--Output:--```--------------------------------------------------------------------------------------------index | item_name | sum_quantity | mean_quanity | maximum_quanity-------|------------------------------|--------------|--------------------|----------------- Int | Text | Int | Double | Int -------|------------------------------|--------------|--------------------|-----------------0 | Chicken Bowl | 761 | 1.0482093663911847 | 3 -1 | Chicken Burrito | 591 | 1.0687160940325497 | 4 -2 | Chips and Guacamole | 506 | 1.0563674321503131 | 4 -3 | Steak Burrito | 386 | 1.048913043478261 | 3 -4 | Canned Soft Drink | 351 | 1.1661129568106312 | 4 -5 | Chips | 230 | 1.0900473933649288 | 3 -6 | Steak Bowl | 221 | 1.04739336492891 | 3 -7 | Bottled Water | 211 | 1.3024691358024691 | 10 -8 | Chips and Fresh Tomato Salsa | 130 | 1.1818181818181819 | 15 -9 | Canned Soda | 126 | 1.2115384615384615 | 4 -```--Full example in `./examples` folder using many of the constructs in the API.--## Installing--### Jupyter notebook-* We have a [hosted version of the Jupyter notebook](https://ulwazi-exh9dbh2exbzgbc9.westus-01.azurewebsites.net/lab) on azure sites. This is hosted on Azure's free tier so it can only support 3 or 4 kernels at a time.-* To get started quickly, use the Dockerfile in the [ihaskell-dataframe](https://github.com/mchav/ihaskell-dataframe) to build and run an image with dataframe integration.-* For a preview check out the [California Housing](https://github.com/mchav/dataframe/blob/main/docs/California%20Housing.ipynb) notebook.--### CLI-* Run the installation script `curl '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/mchav/dataframe/refs/heads/main/scripts/install.sh | sh`-* Download the run script with: `curl --output dataframe "https://raw.githubusercontent.com/mchav/dataframe/refs/heads/main/scripts/dataframe.sh"`-* Make the script executable: `chmod +x dataframe`-* Add the script your path: `export PATH=$PATH:./dataframe`-* Run the script with: `dataframe`---## What is exploratory data analysis?-We provide a primer [here](https://github.com/mchav/dataframe/blob/main/docs/exploratory_data_analysis_primer.md) and show how to do some common analyses.--## Coming from other dataframe libraries-Familiar with another dataframe library? Get started:-* [Coming from Pandas](https://github.com/mchav/dataframe/blob/main/docs/coming_from_pandas.md)-* [Coming from Polars](https://github.com/mchav/dataframe/blob/main/docs/coming_from_polars.md)-* [Coming from dplyr](https://github.com/mchav/dataframe/blob/main/docs/coming_from_dplyr.md)--## Supported input formats-* CSV-* Apache Parquet--## Supported output formats-* CSV--## Future work-* Apache arrow compatability-* Integration with more data formats (SQLite, Postgres, json lines, xlsx).-* Host the whole library + Jupyter lab on Azure with auth and isolation.+For an installation guide and tutorials checkout the [project documentation](https://dataframe.readthedocs.io/) and for an API reference checkout the [hackage documentation](https://hackage-content.haskell.org/package/dataframe-0.3.3.2/docs/DataFrame.html).
dataframe.cabal view
@@ -1,6 +1,6 @@ cabal-version: 2.4 name: dataframe-version: 0.3.3.1+version: 0.3.3.2 synopsis: A fast, safe, and intuitive DataFrame library.
src/DataFrame.hs view
@@ -18,10 +18,32 @@ Example session: +We provide a script that imports the core functionality and defines helpful+macros for writing safe code.+ @--- GHCi quality-of-life:-ghci> :set -XOverloadedStrings -XTypeApplications-ghci> :module + DataFrame as D, DataFrame.Functions as F, Data.Text (Text)+\$ curl --output dataframe \"https:\/\/raw.githubusercontent.com\/mchav\/dataframe\/refs\/heads\/main\/scripts\/dataframe.sh\"+\$ chmod +x dataframe+\$ export PATH=$PATH:$PWD/dataframe+\$ dataframe+Configuring library for fake-package-0...+Warning: No exposed modules+GHCi, version 9.6.7: https:\/\/www.haskell.org\/ghc\/ :? for help+Loaded GHCi configuration from \/tmp\/cabal-repl.-242816\/setcwd.ghci+========================================+ 📦Dataframe+========================================++✨ Modules were automatically imported.++💡 Use prefix 'D' for core functionality.+ ● E.g. D.readCsv \"\/path\/to\/file\"+💡 Use prefix 'F' for expression functions.+ ● E.g. F.sum (F.col \@Int \"value\")++✅ Ready.+Loaded GHCi configuration from ./dataframe.ghci+ghci> @ = Quick start@@ -48,17 +70,18 @@ 9 | longitude | 20640 | 0 | 0 | 844 | Double -- 2) Project & filter-ghci> df1 = D.filter \@Text "ocean_proximity" (== \"ISLAND\") df0 D.|> D.select ["median_house_value", "median_income", "ocean_proximity"]+ghci> :exposeColumn df+ghci> df1 = D.filterWhere (ocean_proximity F.== F.lit \"ISLAND\") df0 D.|> D.select [F.name median_house_value, F.name median_income, F.name ocean_proximity] -- 3) Add a derived column using the expression DSL -- (col types are explicit via TypeApplications)-ghci> df2 = D.derive "rooms_per_household" (F.col \@Double "total_rooms" / F.col \@Double "households") df0+ghci> df2 = D.derive "rooms_per_household" (total_rooms / households) df0 -- 4) Group + aggregate ghci> let grouped = D.groupBy ["ocean_proximity"] df0 ghci> let summary = D.aggregate- [ F.maximum (F.col \@Double "median_house_value") \`F.as\` "max_house_value"]+ [ F.maximum median_house_value \`F.as\` "max_house_value"] grouped ghci> D.take 5 summary -----------------------------------------@@ -92,7 +115,7 @@ __Row ops__ - * @D.filter :: Columnable a => Text -> (a -> Bool) -> DataFrame -> DataFrame@+ * @D.filterWhere :: Expr Bool -> DataFrame -> DataFrame@ * @D.sortBy :: SortOrder -> [Text] -> DataFrame -> DataFrame@ __Column ops__
src/DataFrame/Operations/Subset.hs view
@@ -156,10 +156,9 @@ filterBy :: (Columnable a) => (a -> Bool) -> T.Text -> DataFrame -> DataFrame filterBy = flip filter -{- | O(k) filters the dataframe with a row predicate. The arguments in the function- must appear in the same order as they do in the list.+{- | O(k) filters the dataframe with a boolean expression. -> filterWhere (["x", "y"], func (\x y -> x + y > 5)) df+> filterWhere (F.col @Int x + F.col y F.> 5) df -} filterWhere :: Expr Bool -> DataFrame -> DataFrame filterWhere expr df =