name: shelltestrunner
version: 0.5
category: Testing
synopsis: A tool for testing command-line programs.
description:
.
Run a given program through \"shell\" tests specifed by one or more test
files, each of which can specify: command-line arguments, input, expected
output, expected stderr output, and expected exit code. This was
extracted from the hledger project, inspired by the tests in John
Wiegley's ledger project, and uses test-framework's test runner.
.
Usage:
.
> shelltestrunner [opts] executable testfile1 [testfile2 ...] [-- <test-framework runner opts>]
.
You can pass options through to test-framework's runner; they must go
after -- at the end. You may be able to get a big speedup by running
tests in parallel: try -- -j8.
.
Test file format:
.
> # 0 or more comment lines beginning with #
> -opt1 -opt2 arg1 arg2 # command line args, executable will be prepended
> <<<
> 0 or more lines of input
> >>> [/regexp/]
> [..or 0 or more lines of expected output]
> >>>2 [/regexp/]
> [..or 0 or more lines of expected error output]
> >>>= [/regexp/]
> [..or expected numeric exit code]
.
Each expected field can have either a regular expression match
expression, in which case the test passes if the output is matched, or 0
or more data lines, in which case the output must match these exactly. A
! preceding a /regexp/ negates the match. The regular expression syntax
is that supported by the regexpr library.
.
Apart from the command line, all fields are optional. Only fields
specified in the test will be tested, unless you use the
-i/--implicit-tests flag, which will test for empty stdout, empty stderr,
or 0 exit code where fields are omitted.
.
Issues:
.
- can't put / in a regexp
.
- can't test input/output which does not end with newline
.
- option processing is weak
.
Wishlist:
.
- configurable delimiters
.
- multiple tests per file
.
license: GPL
license-file: LICENSE
author: Simon Michael <simon@joyful.com>
maintainer: Simon Michael <simon@joyful.com>
homepage: http://joyful.com/darcsweb/darcsweb.cgi?r=shelltestrunner
-- bug-reports:
stability: experimental
tested-with: GHC==6.10
cabal-version: >= 1.2
build-type: Simple
extra-tmp-files:
extra-source-files:
executable shelltestrunner
main-is: shelltestrunner.hs
ghc-options: -Wall -threaded
build-depends:
base >= 3 && < 5
,parsec
,process
,HUnit
,test-framework
,test-framework-hunit >= 0.2 && < 0.3
,parseargs >= 0.1 && < 0.2
,regexpr >= 0.5.1