testrunner 0.9 → 0.9.1
raw patch · 5 files changed
+6/−300 lines, 5 files
Files
- Test/Runner/Backends.hs +4/−4
- Test/Runner/Test.hs +0/−83
- homepage.html +0/−52
- testrunner.cabal +2/−1
- using-testrunner.html +0/−160
Test/Runner/Backends.hs view
@@ -63,10 +63,10 @@ runWithArgs args (RunWithQuickCheck t) = do r <- QC.quickCheckWithResult args t return $ case r of- QC.Failure seed size reason _ -> Just (reason ++ " (seed: " ++- show seed ++ ", size: "- ++ show size ++ ")")- _ -> Nothing+ QC.Failure{} -> Just (QC.reason r ++ " (seed: " +++ show (QC.usedSeed r) ++ ", size: "+ ++ show (QC.usedSize r) ++ ")")+ _ -> Nothing -- | HUnit @Test@s can be run by testrunner. instance RunnableTest Test where
− Test/Runner/Test.hs
@@ -1,83 +0,0 @@--- Copyright (C) 2009 Reinier Lamers------ This program is free software; you can redistribute it and/or modify--- it under the terms of the GNU General Public License as published by--- the Free Software Foundation; either version 2, or (at your option)--- any later version.------ This program is distributed in the hope that it will be useful,--- but WITHOUT ANY WARRANTY; without even the implied warranty of--- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the--- GNU General Public License for more details.------ You should have received a copy of the GNU General Public License--- along with this program; see the file COPYING. If not, write to--- the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,--- Boston, MA 02110-1301, USA.--module Main where--import System.Exit ( exitWith, ExitCode(..) )-import Test.HUnit ( runTestTT, Test(..), Counts(..), Assertion, assertEqual,- assertFailure, assertBool )--import Test.Runner.Backends ( TestRunnerTest(..), runWithQuickCheck )-import Test.Runner.Driver ( runTests, Result(Result), runTestsParallel )--main :: IO ()-main = do- counts <- runTestTT tests- if errors counts == 0 && failures counts == 0- then exitWith ExitSuccess- else exitWith (ExitFailure 1)--tests :: Test-tests = TestList [ TestLabel "Checking that empty list of tests does not fail" (TestCase emptyListTest)- , TestLabel "Checking singleton success" (TestCase singletonSuccess)- , TestLabel "Checking singleton failure" (TestCase singletonFailure)- , TestLabel "Checking parallel testing" (TestCase parallelNumber)- ]----- | Verify that testrunner does the right thing when given an empty list-emptyListTest :: Assertion-emptyListTest = do- res <- runTests [] - assertEqual "result for empty list" (Result 0 []) res--singletonSuccess :: Assertion-singletonSuccess = do- res <- runTests [("success", TestRunnerTest True)]- assertEqual "result for singleton success" (Result 1 []) res- resHUnit <- runTests [("HUnit success", TestRunnerTest (TestCase (assertBool "success" True)))]- assertEqual "result for singleton success with HUnit" (Result 1 []) resHUnit- resQC <- runTests [("QuickCheck success", runWithQuickCheck True)]- assertEqual "result for singleton success with QuickCheck" (Result 1 []) resQC- resIO <- runTests [("IO success", TestRunnerTest (return True :: IO Bool))]- assertEqual "result for singleton success with IO" (Result 1 []) resIO--singletonFailure :: Assertion-singletonFailure = do- (Result numPassed fails) <- runTests [("failure", TestRunnerTest False)]- assertEqual "no successes in singleton failure list" 0 numPassed- assertEqual "one failure in singleton failure list" 1 (length fails)- (Result numPassedHU failsHU) <- runTests [("HUnit failure", TestRunnerTest (TestCase (assertFailure "failure")))]- assertEqual "no successes in singleton failure list for HUnit" 0 numPassedHU- assertEqual "one failure in singleton failure list for HUnit" 1 (length failsHU)- (Result numPassedQC failsQC) <- runTests [("QuickCheck failure", runWithQuickCheck False)]- assertEqual "no successes in singleton failure list for QuickCheck" 0 numPassedQC- assertEqual "one failure in singleton failure list for QuickCheck" 1 (length failsQC)- (Result numPassedIO failsIO) <- runTests [("IO failure", TestRunnerTest (return False :: IO Bool))]- assertEqual "no success in singleton failure list for IO" 0 numPassedIO- assertEqual "one failure in singleton failure list for QuickCheck" 1 (length failsIO)---- | Checks that the number of tests executed by the parallel runner is equal to--- the number of tests given (this may easily go wrong in the case of race--- conditions).-parallelNumber :: Assertion-parallelNumber = do- let twoTests = [("success", TestRunnerTest True), ("failure", TestRunnerTest False)]- (Result numPassed fails) <- runTestsParallel 8 (concat $ replicate 500 twoTests)- let numRun = numPassed + length fails- assertEqual "Number of tests run equals number of tests given" 1000 numRun-
− homepage.html
@@ -1,52 +0,0 @@-<?xml version="1.0" encoding="utf-8"?>-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">-<html xmlns="http://www.w3.org/1999/xhtml">- <head>- <title>testrunner</title>- <style type="text/css">- * { font-family: verdana, sans-serif; }- body { margin-left: 20%;- margin-right: 20%; }- img { border: none;- margin: 12px; }- code { font-family: monospace; }- </style>- </head>- <body>- <h1>testrunner</h1>- <p>testrunner is a <a href="http://www.haskell.org/">Haskell</a> library for- running unit tests. It has the- following distinguishing features:</p>- <ul>- <li>It can run unit tests in parallel.</li>- <li>It can run QuickCheck and HUnit tests as well as simple boolean- expressions.</li>- <li>It comes with a ready-made main function for your unit test- executable.</li>- <li>This main function recognizes command-line arguments to select tests- by name and replay QuickCheck tests.</li>- </ul>-- <p>testrunner was spun off the <a href="http://darcs.net/">darcs</a>- project.</p>-- <h2>Using testrunner</h2>- <p>A tutorial for testrunner can be found <a- href="using-testrunner.html">here</a>.</p>- <h2>Getting testrunner</h2>- <p>You can download the 0.9 release of testrunner <a- href="releases/testrunner-0.9.tar.gz">here</a>.</p>- <p>The darcs repository of testrunner is <a- href="http://code.haskell.org/testrunner/"><code>http://code.haskell.org/testrunner/</code></a>.- Thus, use <code>darcs get http://code.haskell.org/testrunner/</code> to check- out the latest code.</p>- <div id="bottombanner">- <a href="http://www.haskell.org/hpc"><img src="hpcbadge.jpg" alt="hpc- badge"/></a>- <a href="http://validator.w3.org/check?uri=referer">- <img src="http://www.w3.org/Icons/valid-xhtml10-blue" alt="Valid XHTML 1.0 Strict" height="31" width="88" />- </a>- </div>- </body>-</html>-
testrunner.cabal view
@@ -1,5 +1,5 @@ Name: testrunner-Version: 0.9+Version: 0.9.1 Category: Testing Synopsis: Easy unit test driver framework Description: testrunner is a framework to run unit tests. It has the@@ -17,6 +17,7 @@ tests by name and replay QuickCheck tests. License: GPL License-file: LICENSE+extra-source-files: README Author: Reinier Lamers <tux_rocker@reinier.de> Maintainer: Reinier Lamers <tux_rocker@reinier.de> Build-Type: Simple
− using-testrunner.html
@@ -1,160 +0,0 @@-<?xml version="1.0" encoding="utf-8"?>-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">-<html xmlns="http://www.w3.org/1999/xhtml">- <head>- <title>Using testrunner</title>- <style type="text/css">- * { font-family: verdana, sans-serif; }- body { margin-left: 20%;- margin-right: 20%; }- code { font-family: monospace; }- pre { font-family: monospace; }- </style>- </head>- <body>- <h1>Using testrunner</h1>- <p>This is a tutorial for the <em>testrunner</em> library. For the details of the exported API, please- consult the <a href="haddock/">haddock- documentation.</a></p>-- <h2>Two ways to use it</h2>-- <p>- There are two ways you can use the testrunner library. The first is to put- all the unit tests of your application in a big list, and then have a- one-line <code>main</code> function for your unit test program that calls- the <code>testRunnerMain</code> function from <code>Test.Runner</code> to- run the unit tests. </p>- <p>If you use testrunner this way, you get a unit test program with a couple- of nice features almost for free. You can replay <a- href="http://www.cs.chalmers.se/~rjmh/QuickCheck/">QuickCheck</a> tests,- select which tests to run, and choose how many tests to run at the same- time from the command line. If you wish to extend the functionality of the unit test- program however, you have to modify the testrunner library.</p>- <p>That is why there is another way to use testrunner: you can use the- functions exported by <code>Test.Runner.Driver</code> and- <code>Test.Runner.Backends</code> to run unit tests. You can tell testrunner- how many tests to run in parallel, or tell it the QuickCheck arguments to- use when running QuickCheck tests. You get back a data structure that you- can examine to see which tests failed and which succeeded. This way of using- testrunner is not further described in this document, but see the <a- href="haddock/Test-Runner-Driver.html">haddock</a>.</p>-- <h2>Using testrunner the first way, using- <code>testRunnerMain</code></h2>- <p>Say you have a program called <code>hello</code>, which contains the- simple source file <code>Hello.hs</code> shown here:</p>- <pre>- module Hello where-- helloWorld :: String- helloWorld = hello "world"-- hello :: String -> String- hello s = "hello " ++ s- </pre>-- <p>Now we want to define unit tests for this module. Let's say we come up- with the following. I realize that they are silly, but this is- not a guide to testing the right properties, this is a guide to using- testrunner. Here is the test code, in a file <code>Test.hs</code>:</p>- <pre>- module Main where-- import Test.HUnit- import Test.QuickCheck-- import Hello-- -- use HUnit to assert that helloWorld produces "hello world"- hunitTest :: Test- hunitTest = TestCase $ do- assertEqual "hello world" "hello world" helloWorld-- -- use QuickCheck to check the length of hello's result- helloLength :: String -> Bool- helloLength s = length (hello s) == length "hello " + length s-- -- A simple boolean expression that states that hello of an empty string is- -- "hello"- helloEmpty :: Bool- helloEmpty = hello "" == "hello "- </pre>- <p>The only thing lacking from this test module is a main function. Of- course, you could easily write your own, but the simple version would not- support parallel test execution, selecting unit tests to execute, or running- the QuickCheck test with the same random sample as a previous run.</p>-- <p>- testrunner lets you write your main function more concise and gives you- those nice properties for free. Here is <code>Test.hs</code> again, with the- testrunner-based main function:- </p>- <pre>- module Main where-- import Test.HUnit- import Test.QuickCheck- import Test.Runner-- import Hello-- -- use HUnit to assert that helloWorld produces "hello world"- hunitTest :: Test- hunitTest = TestCase $ do- assertEqual "hello world" "hello world" helloWorld-- -- use QuickCheck to check the length of hello's result- helloLength :: String -> Bool- helloLength s = length (hello s) == length "hello " + length s-- -- A simple boolean expression that states that hello of an empty string is- -- "hello"- helloEmpty :: Bool- helloEmpty = hello "" == "hello "-- tests :: [(String, TestRunnerTest)]- tests = [("helloWorld value", TestRunnerTest hunitTest),- ("hello length", runWithQuickCheck helloLength),- ("value of hello applied to empty string", TestRunnerTest helloEmpty)]- main :: IO ()- main = testRunnerMain tests- </pre>- <p>Here, we put all the unit tests in a list of type <code>[(String,- TestRunnerTest)]</code>, where the first element of every tuple is the- name of the test. We use the <code>TestRunnerTest</code> constructor- to create a <code>TestRunnerTest</code> from HUnit tests and boolean- expressions, and use <code>runWithQuickCheck</code> to turn QuickCheck- <code>Testable</code>s into <code>TestRunnerTest</code>s. Then our main- function is a one-liner that calls <code>testRunnerMain</code> on this- list.</p>-- <p>Now your powerful unit test program is ready! For example, try the- commands:</p>- <pre>- ghc --make -threaded Test.hs -o test- ./test- ./test -m length- ./test -r '1387922338 2147483372,86'- ./test -r '1387922338 2147483372,86' -m length- ./test -j 3 +RTS -N3- </pre>- <p>The first command compiles the <code>Test.hs</code> file to an executable- named <code>test</code>. The second command just runs all the tests and- reports the result. The second command runs only the test whose name matches- the regular expression 'length'. The third command tries the QuickCheck test- (<code>helloLength</code>) with the random seed of 1387922338 2147483372 and- size 86, a combination that makes a lot of darcs unit tests fail. The fourth- command runs only the QuickCheck tests like the second, and runs it with the- random seed and size of the third command. The fifth command runs all the- unit tests in parallel.</p>-- <p>You may notice that screen output becomes garbled when running tests in- parallel. This is partly due to the QuickCheck API, and that is why it is- not fixed. The report at the end (the single line "3 tests passed" in this- example) is printed after the worker threads have- quit, and thus will always be readable.</p>-- <p>That's it folks, enjoy!</p>- </body>-</html>