# hanalyze-frame
The **data I/O layer** of [`hanalyze`](../README.md). It adopts
Hackage's `dataframe` as the single data representation and covers everything
from **loading dirty data** to **tidyverse-style wrangling**.
It depends on `hanalyze-core` plus `dataframe-*` / `cassava` /
`regex-tdfa` and friends. Together with `-bayes` it sits directly on top of
core, and the upper layers (`-models` / `-design` / `-viz`) all assume the data
representation defined here.
## Main modules (14 in total)
### Loading (`Hanalyze.DataIO.*`)
| Module | Role |
|---|---|
| `DataIO.CSV` | CSV / TSV / SSV loaders returning a `DataFrame` directly. `loadAuto` dispatches on the extension; `loadAutoSafe` is the defensive variant returning `Either` plus a log |
| `DataIO.Sniff` | Guesses delimiter, comment marker, header presence and NA tokens from the first 8 KB |
| `DataIO.Health` | Flags suspicious patterns in a loaded `DataFrame` as warning codes W001–W008 |
| `DataIO.Clean` | A per-column cleaning DSL that turns Health warnings into numeric rules |
| `DataIO.Log` | Structured warnings shared by loaders and preprocessing (`LogEntry` / `LogReport`) |
| `DataIO.External` | Parquet / JSON loaders (via `dataframe`) |
| `DataIO.Convert` | Safe extraction of numeric / text columns from a `DataFrame` into `Vector` |
### Wrangling (`Hanalyze.Data.*` / `DataIO.*`)
| Module | Role |
|---|---|
| `Data.Wrangle` | dplyr-style `summarise` / `mutate` / `groupBy`, `DataFrame` in and out. Designed symmetrically with hgg's pipe notation |
| `Data.Transform` | dplyr-style ranking / offsets / cumulatives / binning as pure `[a] -> [b]` |
| `Data.Factor` | forcats-style factor type and level operations (`fct_*`) |
| `Data.Strings` | stringr-style pure Text operations (`str_*`) |
| `Data.ColumnSource` | Minimal "column name → numeric column" abstraction (plot-independent) |
| `DataIO.Reshape` | Reshape operations `dataframe` lacks (pivotWider / oneHot / lag & lead / rolling) |
| `DataIO.Preprocess` | Missing-value detection, removal and imputation / column selection / derived columns / melt |
## Using it standalone
```cabal
build-depends: hanalyze-frame, dataframe-core
```
```haskell
{-# LANGUAGE OverloadedStrings #-}
import Hanalyze.DataIO.CSV (loadAuto)
import Hanalyze.Data.Wrangle
import DataFrame.Operators ((|>))
main = do
Right df <- loadAuto "flights.csv" -- IO (Either ParseError DataFrame)
let out = df |> groupBy ["month"]
|> summarise [ "mean" =: meanOf "dep_delay"
, "q95" =: quantileOf 0.95 "dep_delay"
, "n" =: nOf ]
print out
-- month | mean | q95 | n
-- ------|--------------------|--------------------|----
-- 1 | 4.5 | 11.549999999999999 | 3
-- 2 | 11.166666666666666 | 23.25 | 3
```
Aggregators drop NAs by default (dplyr's `na.rm = TRUE`), and groups come out
in ascending key order.
Normally you would just depend on the umbrella package `hanalyze` and
reach these through `import Hanalyze`. Naming a layer directly is only
worth it when you want to minimise dependencies.
## Related docs
- Dirty-data defence (W001–W008 / auto-sniff / clean DSL):
[docs/io/01-dirty-data.md](../docs/io/01-dirty-data.md)
- Reshape (pivot_wider / one-hot / lag-lead / rolling):
[docs/io/02-reshape.md](../docs/io/02-reshape.md)
- Long-form regrid: [docs/io/03-regrid.md](../docs/io/03-regrid.md)
- The unified `df |-> model` fit API: [docs/io/04-fit-api.md](../docs/io/04-fit-api.md)
← [repository README](../README.md)