name: shelltestrunner
version: 0.7
category: Testing
synopsis: A tool for testing command-line programs.
description:
.
Run a command-line program through \"shell tests\" defined in one or
more test files. Each test specifies command-line arguments, some input,
and expected output, stderr output and/or exit status.
We use test-framework's test runner, so can run tests in parallel.
shelltestrunner was inspired by the tests in John Wiegley's ledger project.
.
Usage:
.
> shelltestrunner [FLAGS] EXECUTABLE [TESTFILES ...]
>
> -? --help[=FORMAT] Show usage information (optional format)
> -V --version Show version information
> -v --verbose Higher verbosity
> -q --quiet Lower verbosity
> -d --debug show debug messages
> --debug-parse show parsing debug messages and stop
> -i --implicit=none|exit|all provide implicit tests (default=exit)
.
Any unrecognised options will be passed through to test-framework's runner.
You may be able to get a big speedup by running tests in parallel: try -j16.
.
A test file contains one or more shell tests, each of which looks like this:
.
> # optional comment lines beginning with #
> -opt1 -opt2 arg1 arg2 # one line of command line args, executable will be prepended
> <<<
> 0 or more lines of input
> >>> [/regexp/] [optional comment beginning with #]
> [..or 0 or more lines of expected output]
> >>>2 [/regexp/] [optional comment]
> [..or 0 or more lines of expected error output]
> >>>= [expected exit status or /regexp/] [optional comment]
> # optional comment lines
.
The expected output and expected error output fields can have either a
regular expression match expression, in which case the test passes if the
output is matched, or 0 or more following data lines, in which case the
output must match these exactly. The expected exit status field can have
either a numeric exit code or a regular expression match expression. A !
preceding a an exit code or \/regexp\/ negates the match. The regular
expression syntax is that of the pcre-light library with the dotall flag.
.
The command line args field is required, all others are optional. By
default there is an implicit test for exit status = 0, but no implicit
test for stdout or stderr. You can change this with the
-i/--implicit-tests flag.
.
Some issues to be aware of:
.
- uncompiled haskell scripts can't be tested due to http://hackage.haskell.org/trac/ghc/ticket/3890
.
- unparseable test files and uncompilable regexps are counted as test failures
.
license: GPL
license-file: LICENSE
author: Simon Michael <simon@joyful.com>
maintainer: Simon Michael <simon@joyful.com>
homepage: http://joyful.com/darcsweb/darcsweb.cgi?r=shelltestrunner
bug-reports: mailto:simon@joyful.com
stability: beta
tested-with: GHC==6.10
cabal-version: >= 1.6
build-type: Simple
extra-tmp-files:
extra-source-files:
executable shelltestrunner
main-is: shelltestrunner.hs
ghc-options: -Wall -threaded
build-depends:
base >= 3 && < 5
,parsec
,process
,HUnit
,test-framework
,test-framework-hunit >= 0.2
,cmdargs >= 0.1
,pcre-light >= 0.3.1
source-repository head
type: darcs
location: http://joyful.com/repos/hledger