diff --git a/README.md b/README.md new file mode 100644 --- /dev/null +++ b/README.md @@ -0,0 +1,54 @@ +# What is eddie? + +A command line tool for running Haskell functions over text files. + +It is also useful for prototyping Haskell text processing functions +that you will later copy into a program. + +# Why is eddie? + +This project actually started in response to a code bounty for a +clojure shell scripting tool. That resulted in a specification for the +requested program that had all the features of eddie, and then +some. However, clojure code embedded on the command line looks clumsy, +so the project was shelved. + +In haskell, functional expression are much simpler and more +elegant. In particular, in looking for haskell command line tools, I +found , which +convinced me that the original design would be not merely usable, but +elegant if the expressions were done in haskell. Hence, eddie. + +# Why not `ghc -e`? + +Eddie adds features to make using it for shell scripting easier: + + * When given file arguments, eddie feeds them to your function. + * Eddie can easily add modules to the namespace you use. + * Eddie has options for processing things a line or file at a time. + * Eddie will use binary file IO methods when asked to. + +# More information + +The best place to start is the manual. That's available in the source +tree in both man format in `eddie.1`, and in html in `eddie.html`. +The latter is also available [online](eddie.html). + +# Installation + +Install or build with cabal as usual. + +# Testing + +Test use the haskell shelltestrunner package (can be installed with +cabal). Run `shelltest tests` in the source directory to run +the tests with the currently installed eddie. Use +`shelltest tests -w eddie.hs` to run them using the current +source. Use `shelltest tests -w dist/build/eddie/eddie` to +run them with the compiled binary. + +# Documentation + +The documenation is built with `pandoc` from +`eddie.md`. Use the `-s` flag, and convert from +pandoc's markdown to your chosen format. diff --git a/eddie.1 b/eddie.1 --- a/eddie.1 +++ b/eddie.1 @@ -1,179 +1,134 @@ -.TH eddie 1 +.TH "eddie" "1" "" "" "" .SH NAME +.PP eddie \- run haskell filters from the command line .SH SYNOPSYS -.B eddie -.RI [ options ] -.RI [ expr ] -.I file ... +.PP +\f[B]eddie\f[] [ \f[I]options\f[] ] [ \f[I]expr\f[] ] \f[I]file\f[] ... .SH DESCRIPTION -.B eddie -evalutes the provided -.I expression -on either the contents of the -.I file -arguments concatenated together, -or standard input if no file arguments are present. .PP -The -.BR \-\-line , -.B \-\-file -and -.B \-\-list -options can be used to cause -.I expression -to be used to process each line, each file, -or a list of files or lines respectively. +\f[B]eddie\f[] evalutes the provided \f[I]expression\f[] on either the +contents of \f[I]the file\f[] arguments concatenated together, or +standard input if no *file arguments are present. .PP -The prelude, Data.List and Data.Char modules are available for building -expressions. Other modules may be added with the -.B \-\-Modules -and -.B \-\-modules -options. +The \f[B]\-\-line\f[], \f[B]\-\-file\f[] and \f[B]\-\-list\f[] options +can be used to cause \f[I]expression\f[] to be used to process each +line, each file, or a list of files or lines respectively. +.PP +The prelude, \f[C]Data.List\f[] and \f[C]Data.Char\f[] modules are +available for building expressions. +Other modules may be added with the \f[B]\-\-Modules\f[] and +\f[B]\-\-modules\f[] options. .SH OPTIONS -.IP \-\-expr -The -.B \-\-expr -.I expr -(short form -.B \-e -.IR expr ) -option concatenates it's value to the -haskell expression being evaluated with a newline separator. -Multiple occurrences can be used to build up a -multi-line expression. +.TP +.B \f[B]\-\-expr\f[] +The \f[B]\-\-expr\f[] \f[I]expr\f[] (short form \f[B]\-e\f[] +\f[I]expr\f[] ) option concatenates it\[aq]s value to the haskell +expression being evaluated with a newline separator. +Multiple occurrences can be used to build up a multi\-line expression. .RS .PP -If no -.B \-e -.I expr -option is present, the first non-flag argument will be used for -the haskell expression. -.RE 1 -.IP \-\-line -The -.B \-\-line -(short form -.BR \-l ) -option causes -.B eddie -to process input line at a time. The -.I expr -will be run on each line and the results concatenated together -(using unlines). +If no \f[B]\-e\f[] \f[I]expr\f[] option is present, the first non\-flag +argument will be used for the haskell expression. +.RE +.TP +.B \f[B]\-\-line\f[] +The \f[B]\-\-line\f[] (short form \f[B]\-l\f[]) option causes +\f[B]eddie\f[] to process input line at a time. +The \f[I]expr\f[] will be run on each line and the results concatenated +together (using unlines). .RS .PP The command -.RS -.PP -.B eddie \-l -.I expr file ... -.RE +.IP +.nf +\f[C] +eddie\ \-l\ *expr\ file*\ ... +\f[] +.fi .PP is equivalent to the command -.RS -.PP -.B eddie -"unlines . map -.IR expr . -lines" -.I file ... -. -.RE 1 -.IP \-\-file -The -.B \-\-file -(short form -.BR \-f ) -causes each file to be processed individually. +.IP +.nf +\f[C] +eddie\ "unlines\ .\ map\ *expr*\ \ .\ lines"\ *file*\ ... +\f[] +.fi +.RE +.TP +.B \f[B]\-\-file\f[] +The \f[B]\-\-file\f[] (short form \f[B]\-f\f[]) causes each file to be +processed individually. .RS .PP -Note that the -.B \-l -and -.B \-f -options cannot be used together. -.RE 1 -.IP \-\-list -The -.B \-\-list -(short form -.BR \-L ) -option causes -.B eddie -to pass the expression a list of lines or files. -This requires one of the -.B \-l -or -.B \-f -options be used. +Note that the \f[B]\-l\f[] and \f[B]\-f\f[] options cannot be used +together. +.RE +.TP +.B \f[B]\-\-list\f[] +The \f[B]\-\-list\f[] (short form \f[B]\-L\f[]) option causes +\f[B]eddie\f[] to pass the expression a list of lines or files. +This requires one of the \f[B]\-l\f[] or \f[B]\-f\f[] options be used. .RS .PP The command -.RS -.PP -.B eddie \-lL -.I expr file ... -.RE +.IP +.nf +\f[C] +eddie\ \-lL\ *expr\ file*\ ... +\f[] +.fi .PP is equivalent to the command -.RS -.PP -.B eddie -"unlines . -.I expr -\&. lines" -.I file ... -. -.RE 1 -.IP \-\-binary -The -.B \-\-binary -(short form -.BR \-b ) -option switches the output mode to binary. +.IP +.nf +\f[C] +eddie\ "unlines\ .\ *expr*\ .\ lines"\ filei\ ... +\f[] +.fi +.RE +.TP +.B \f[B]\-\-binary\f[] +The \f[B]\-\-binary\f[] (short form \f[B]\-b\f[]) option switches the +output mode to binary. It also disables the maybe newline processing. -Normally -.B eddie -will output a final newline if the last character -of the input is not a newline to place the prompt properly. -If the -.B \-b -option is used, the final newline will not be output. -.RE 1 -.IP \-\-modules -The -.B \-\-modules -.I name -(short form -.B \-m -.IR name ) -option is used to import module -.I name -into haskell before evaluating the expression. -.IP \-\-Modules -The -.B \-\-Modules -.I name,as -(short form -.B \-M -.IR name,as ) -option imports the -.I name -module using a qualified import with an as clause, -with -.I as +Normally \f[B]eddie\f[] will output a final newline if the last +character of the input is not a newline to place the prompt properly. +If the \f[B]\-b\f[] option is used, the final newline will not be +output. +.RS +.RE +.TP +.B \f[B]\-\-modules\f[] +The \f[B]\-\-modules\f[] \f[I]name\f[] (short form \f[B]\-m\f[] +\f[I]name\f[]) option is used to import module \f[I]name\f[] into +haskell before evaluating the expression. +.RS +.RE +.TP +.B \f[B]\-\-Modules\f[] +The \f[B]\-\-Modules\f[] \f[I]name\f[],\f[I]as\f[] (short form +\f[B]\-M\f[] \f[I]name\f[],\f[I]as\f[]) option imports the \f[I]name\f[] +module using a qualified import with an as clause, with \f[I]as\f[] being the value for the as clause. +.RS +.RE .SH DIAGNOSTICS -If the command runs with no problems, eddie will exit with a status of 0. -If there are problems with the options, such that the expression is never -evaluated, then eddie will exit with a status of 1. -If haskell returns an error - probably a compilation problem - then +.PP +If the command runs with no problems, eddie will exit with a status of +0. +If there are problems with the options, such that the expression is +never evaluated, then eddie will exit with a status of 1. +If haskell returns an error \- probably a compilation problem \- then eddie will exit with the status of 2. .SH SEE ALSO -The wiki at for examples. +.PP +The wiki at +for examples. .SH BUGS -See the issues list at . +.PP +See the issues list at +. .SH AUTHOR -Mike Meyer (mwm@mired.org) +.PP +Mike Meyer diff --git a/eddie.cabal b/eddie.cabal --- a/eddie.cabal +++ b/eddie.cabal @@ -1,5 +1,5 @@ Name: eddie -Version: 0.5 +Version: 0.5.1 Synopsis: Command line file filtering with haskell Description: A tool to let you use short haskell expressions to filter files at the command line. @@ -10,14 +10,20 @@ Copyright: (c) 2011 Mike Meyer Maintainer: mwm@mired.org Build-type: Simple -Homepage: http://eddie.googlecode.com/ +Homepage: http://chiselapp.com/user/mwm/repository/eddie/ Cabal-version: >=1.8 -Extra-source-files: eddie.1 tests/*.test tests/*.data +Extra-source-files: eddie.md eddie.1 eddie.html README.md + tests/*.test tests/*.data Executable eddie Main-is: eddie.hs - Build-Depends: hint >= 0.3.3.2, base >= 4.3.1 && < 5, cmdargs >= 0.7 + Build-Depends: hint >= 0.3.3.2, base >= 4.2.0 && < 5, cmdargs >= 0.7 Source-repository head - Type: hg - Location: http://eddie.googlecode.com/hg/ + Type: fs + Location: https://chiselapp.com/user/mwm/repository/eddie/ + +Source-repository this + Type: fs + Location: https://chiselapp.com/user/mwm/repository/eddie/ + Tag: 0.5.1 diff --git a/eddie.hs b/eddie.hs --- a/eddie.hs +++ b/eddie.hs @@ -1,10 +1,11 @@ #!/usr/bin/env runghc {-# LANGUAGE DeriveDataTypeable #-} {-# LANGUAGE GADTs #-} +{-# LANGUAGE NamedFieldPuns #-} --- import Data.ByteString.Lazy (readFile) import Language.Haskell.Interpreter - (setImportsQ, interpret, runInterpreter, as, MonadInterpreter) + (setImportsQ, interpret, runInterpreter, as, MonadInterpreter, + InterpreterError (..), GhcError (..)) import Data.IORef (newIORef, readIORef, writeIORef) import System.Environment (getArgs, getProgName) import System.IO @@ -31,38 +32,49 @@ when (binary opts) $ hSetBinaryMode stdout True fun <- runInterpreter $ makeFun opts case fun of - Left e -> do hPutStrLn stderr $ name ++ ": Error: " ++ show e + Left e -> do hPutStrLn stderr $ name ++ ": Error: " ++ interpreterErrorMsg e exitWith (ExitFailure 2) Right f -> withFiles (files opts) opener (outputFunc . f) + +interpreterErrorMsg :: InterpreterError -> String +interpreterErrorMsg err = case err of + UnknownError msg -> "Unknown error: " ++ msg + WontCompile ghcErrors -> "GHC errors:\n" ++ concatMap errMsg ghcErrors + NotAllowed msg -> "Not allowed: " ++ msg + GhcException msg -> "GHC exceptions: " ++ msg + + putStrMaybeLn :: Bool -> String -> IO () -putStrMaybeLn binary val = +putStrMaybeLn binary val = (if binary || last val == '\n' then putStr else putStrLn) val -makeFun :: MonadInterpreter m => Eddie -> m ([String] -> String) +makeFun :: MonadInterpreter m => Eddie -> m ([(FilePath, String)] -> String) makeFun opts = do setImportsQ (asModules opts) eval (head (expr opts)) mode where mode | line opts && list opts = Mode (as :: [String] -> [String]) id - (lines . concat) unlines - | line opts = Mode (as :: String -> String) map - (lines . concat) unlines - | file opts && list opts = Mode (as :: [String] -> [String]) id - id concat - | file opts = Mode (as :: String -> String) map id concat - | otherwise = Mode (as :: String -> String) id concat id + (lines . concat . map snd) unlines + | line opts = Mode (as :: String -> String) map + (lines . concat . map snd) unlines + | file opts && list opts = Mode (as :: [String] -> [String]) id + (map snd) concat + | file opts = Mode (as :: String -> String) map (map snd) concat + | otherwise = Mode (as :: String -> String) id (concat . map snd) + id data Mode where Mode :: (Typeable a, Typeable b) - => (a -> b) -- witness to `interpret` - -> ((a -> b) -> c -> d) -- application of interpreted function - -> ([String] -> c) -- preprocess - -> (d -> String) -- postprocess + => (a -> b) -- witness to `interpret` + -> ((a -> b) -> c -> d) -- application of interpreted function + -> ([(FilePath, String)] -> c) -- preprocess + -> (d -> String) -- postprocess -> Mode -eval :: MonadInterpreter m => String -> Mode -> m ([String] -> String) +eval :: MonadInterpreter m => String + -> Mode -> m ([(FilePath, String)] -> String) eval s (Mode f t bra ket) = liftM ((ket .) . (. bra) . t) $ interpret s f -- an even lazier version of withFile (courtesy of Heinrich Apfelmus) @@ -86,18 +98,19 @@ return (c:cs) -withFiles :: [FilePath] -> (FilePath -> IOMode -> IO Handle) -> - ([String] -> IO a) -> IO a -withFiles [] o f = withFile (return stdin) (f . (:[])) -withFiles [x] o f = withFile (o x ReadMode) (f . (:[])) +withFiles :: [FilePath] -> (FilePath -> IOMode -> IO Handle) -> + ([(FilePath, String)] -> IO a) -> IO a +withFiles [] o f = withFile (return stdin) (\s -> f [("", s)]) +withFiles [x] o f = withFile (o x ReadMode) (\s -> f [(x, s)]) withFiles (x:xs) o f = withFile (o x ReadMode) $ \s -> - let f' t = f (s:t) in withFiles xs o f' + let f' t = f ((x, s):t) in withFiles xs o f' -- argument processing data Eddie = Eddie { line :: Bool, file :: Bool, list :: Bool, + withNames :: Bool, binary :: Bool, expr :: [String], files :: [String], @@ -106,7 +119,7 @@ } deriving (Show, Data, Typeable) parseOpts :: Eddie -> Either String Eddie -parseOpts opts = +parseOpts opts = let es = expr opts fs = files opts e:fs' = case (es, fs) of @@ -121,16 +134,18 @@ "--help for options"] else Right $ opts {expr = [e], asModules = mods, files = fs' } - + eddie = Eddie {line = False &= help "Process one line at a time (conflicts with --file)", file = False &= help "Process files individually (requires at least one file name)", - list = False &= help "Process the list of files/lines (requires --line or --file)" &= name "L", - binary = False &= help "Process a binary file", + list = False &= help "Process the list of files/lines (requires --line or --file)" &= name "L", + withNames = False &= help "Prepends file name to output lines (conflicts with --list, requires --line or --file and at least one file name)" + &= explicit &= name "name" &= name "n", + binary = False &= help "Process binary data", expr = [] &= help "Line of expression to evaluate" &= name "e", modules = ["Prelude", "Data.List", "Data.Char"] &= help "Modules to import for expr", asModules = [] &= help "Modules to import qualified" &= explicit &= name "M" &= name "Modules", - files = [] &= args} - &= summary "eddie 0.5" &= details ["Haskell for shell scripts."] - + files = [] &= args} + &= summary "eddie 0.6" &= details ["Haskell for shell scripts."] + diff --git a/eddie.html b/eddie.html new file mode 100644 --- /dev/null +++ b/eddie.html @@ -0,0 +1,70 @@ + + + + + + + eddie(1) + + + + +

NAME

+

eddie - run haskell filters from the command line

+

SYNOPSYS

+

eddie [ options ] [ expr ] file ...

+

DESCRIPTION

+

eddie evalutes the provided expression on either the contents of the file arguments concatenated together, or standard input if no *file arguments are present.

+

The --line, --file and --list options can be used to cause expression to be used to process each line, each file, or a list of files or lines respectively.

+

The prelude, Data.List and Data.Char modules are available for building expressions. Other modules may be added with the --Modules and --modules options.

+

OPTIONS

+
+
--expr
+

The --expr expr (short form -e expr ) option concatenates it's value to the haskell expression being evaluated with a newline separator. Multiple occurrences can be used to build up a multi-line expression.

+

If no -e expr option is present, the first non-flag argument will be used for the haskell expression.

+
+
--line
+

The --line (short form -l) option causes eddie to process input line at a time. The expr will be run on each line and the results concatenated together (using unlines).

+

The command

+
eddie -l *expr file* ...
+

is equivalent to the command

+
eddie "unlines . map *expr*  . lines" *file* ...
+
+
--file
+

The --file (short form -f) causes each file to be processed individually.

+

Note that the -l and -f options cannot be used together.

+
+
--list
+

The --list (short form -L) option causes eddie to pass the expression a list of lines or files. This requires one of the -l or -f options be used.

+

The command

+
eddie -lL *expr file* ...
+

is equivalent to the command

+
eddie "unlines . *expr* . lines" filei ...
+
+
--binary
+
The --binary (short form -b) option switches the output mode to binary. It also disables the maybe newline processing. Normally eddie will output a final newline if the last character of the input is not a newline to place the prompt properly. If the -b option is used, the final newline will not be output. +
+
--modules
+
The --modules name (short form -m name) option is used to import module name into haskell before evaluating the expression. +
+
--Modules
+
The --Modules name,as (short form -M name,as) option imports the name module using a qualified import with an as clause, with as being the value for the as clause. +
+
+

DIAGNOSTICS

+

If the command runs with no problems, eddie will exit with a status of 0. If there are problems with the options, such that the expression is never evaluated, then eddie will exit with a status of 1. If haskell returns an error - probably a compilation problem - then eddie will exit with the status of 2.

+

SEE ALSO

+

The wiki at https://chiselapp.com/user/mwm/repository/eddie/wcontent for examples.

+

BUGS

+

See the issues list at https://chiselapp.com/user/mwm/repository/eddie/rptview?rn=1.

+

AUTHOR

+

Mike Meyer

+ + diff --git a/eddie.md b/eddie.md new file mode 100644 --- /dev/null +++ b/eddie.md @@ -0,0 +1,119 @@ +% eddie(1) + +NAME +==== + +eddie \- run haskell filters from the command line + +SYNOPSYS +======== + + +**eddie** [ *options* ] [ *expr* ] *file* ... + +DESCRIPTION +=========== + + +**eddie** evalutes the provided *expression* on either the contents of +*the file* arguments concatenated together, or standard input if no +*file arguments are present. + +The **--line**, **--file** and **--list** options can be used to cause +*expression* to be used to process each line, each file, or a list of +files or lines respectively. + +The prelude, `Data.List` and `Data.Char` modules are available for building +expressions. Other modules may be added with the **--Modules** and +**--modules** options. + +OPTIONS +======= + +**--expr** +: The **--expr** *expr* (short form **-e** *expr* ) option + concatenates it's value to the haskell expression being evaluated + with a newline separator. Multiple occurrences can be used to + build up a multi-line expression. + + If no **-e** *expr* option is present, the first non-flag argument + will be used for the haskell expression. + +**--line** +: The **--line** (short form **-l**) option causes **eddie** to + process input line at a time. The *expr* will be run on each line + and the results concatenated together (using unlines). + + The command + ``` + eddie -l *expr file* ... + ``` + is equivalent to the command + ``` + eddie "unlines . map *expr* . lines" *file* ... + ``` + +**--file** +: The **--file** (short form **-f**) causes each file to be processed + individually. + + Note that the **-l** and **-f** options cannot be used together. + +**--list** +: The **--list** (short form **-L**) option causes **eddie** to pass + the expression a list of lines or files. This requires one of the + **-l** or **-f** options be used. + + The command + + ``` + eddie -lL *expr file* ... + ``` + is equivalent to the command + + ``` + eddie "unlines . *expr* . lines" filei ... + ``` + +**--binary** +: The **--binary** (short form **-b**) option switches the output mode + to binary. It also disables the maybe newline processing. + Normally **eddie** will output a final newline if the last + character of the input is not a newline to place the prompt + properly. If the **-b** option is used, the final newline will + not be output. + +**--modules** +: The **--modules** *name* (short form **-m** *name*) option is used + to import module *name* into haskell before evaluating the + expression. + +**--Modules** +: The **--Modules** *name*,*as* (short form **-M** *name*,*as*) option + imports the *name* module using a qualified import with an as + clause, with *as* being the value for the as clause. + +DIAGNOSTICS +=========== + +If the command runs with no problems, eddie will exit with a status of 0. +If there are problems with the options, such that the expression is never +evaluated, then eddie will exit with a status of 1. +If haskell returns an error - probably a compilation problem - then +eddie will exit with the status of 2. + +SEE ALSO +======== + +The wiki at +for examples. + +BUGS +==== + +See the issues list at . + +AUTHOR +====== + +Mike Meyer diff --git a/tests/error.test b/tests/error.test --- a/tests/error.test +++ b/tests/error.test @@ -1,3 +1,7 @@ eddie the.lines ->>>2 /Error: / +>>>2 /GHC errors:/ >>>= 2 + +eddie show.length ./- +>>>2 /does not exist/ +>>>= 1