test-framework 0.1.2 → 0.2.0
raw patch · 5 files changed
+8/−324 lines, 5 filesdep −QuickCheckdep ~HUnitdep ~base
Dependencies removed: QuickCheck
Dependency ranges changed: HUnit, base
Files
- README.textile +3/−10
- Test/Framework/Example.lhs +0/−110
- Test/Framework/Providers/HUnit.hs +0/−56
- Test/Framework/Providers/QuickCheck.hs +0/−96
- test-framework.cabal +5/−52
README.textile view
@@ -14,19 +14,12 @@ </pre> </code> -If you want to build the example, to check it's all working: - -<pre> -<code>runghc Setup.lhs configure -fexample -runghc Setup.lhs build -dist/build/test-framework-example/test-framework-example -</pre> -</code> +If you want to build the example, in order to check it's all working you will need to "download it from GitHub":http://github.com/batterseapower/test-framework/tree/master/example/Test/Framework/Example.lhs. Once you have it follow the instructions in the file itself as to how to build it. h2. Description -A test framework, with built-in support for "HUnit":http://hackage.haskell.org/cgi-bin/hackage-scripts/package/HUnit and "QuickCheck":http://hackage.haskell.org/cgi-bin/hackage-scripts/package/QuickCheck tests. The main benefit of using this framework is that you get a nice console based test runner with the following features: +An extensible test framework, with support for "HUnit":http://hackage.haskell.org/cgi-bin/hackage-scripts/package/HUnit and "QuickCheck":http://hackage.haskell.org/cgi-bin/hackage-scripts/package/QuickCheck tests through the "test-framework-hunit":http://hackage.haskell.org/cgi-bin/hackage-scripts/package/test-framework-hunit and "test-framework-quickcheck":http://hackage.haskell.org/cgi-bin/hackage-scripts/package/test-framework-quickcheck packages. The main benefit of using this framework is that you get a nice console based test runner with the following features: * Run tests in parallel but report results in a deterministic order (to aid diff-based analysis of test output) * Progress reporting for individual QuickCheck properties and for the whole suite being run @@ -39,7 +32,7 @@ h2. Example -An example testsuite is provided in the package, which you can build by supplying @-fexample@ to the Cabal @configure@ step as described above. You can also view the most recent version online at "GitHub":http://github.com/batterseapower/test-framework/tree/master/Test/Framework/Example.lhs. +You can view the most recent version of the testsuite online at "GitHub":http://github.com/batterseapower/test-framework/tree/master/example/Test/Framework/Example.lhs, as described in the "Installing" section. The example is not currently available via Hackage. There are two essential components to getting running with the test framework: setting up the tests to be run, and making the program run the tests in the provided console test runner.
− Test/Framework/Example.lhs
@@ -1,110 +0,0 @@-== RUNNING ==--ghc -package test-framework -threaded Example.lhs -o Example-./Example --maximum-generated-tests=5000 +RTS -N2---== ATTRIBUTION ==--Tthe example properties come from the parallel QuickCheck driver (pqc),-see http://code.haskell.org/~dons/code/pqc/. The BSD license is repeated-below, per the licensing conditions of pqc.--== LICENSING ==--Copyright Don Stewart 2006.--All rights reserved.--Redistribution and use in source and binary forms, with or without-modification, are permitted provided that the following conditions are-met:-- * Redistributions of source code must retain the above copyright- notice, this list of conditions and the following disclaimer.-- * Redistributions in binary form must reproduce the above- copyright notice, this list of conditions and the following- disclaimer in the documentation and/or other materials provided- with the distribution.-- * Neither the name of Don Stewart nor the names of other- contributors may be used to endorse or promote products derived- from this software without specific prior written permission.--THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS-"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT-LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR-A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT-OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,-SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT-LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,-DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY-THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE-OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.--\begin{code}--import Test.Framework-import Test.Framework.Providers.HUnit-import Test.Framework.Providers.QuickCheck--import Test.QuickCheck-import Test.HUnit--import Data.List---main = defaultMain tests--tests = [- testGroup "Sorting Group 1" [- testProperty "sort1" prop_sort1,- testProperty "sort2" prop_sort2,- testProperty "sort3" prop_sort3- ],- testGroup "Sorting Group 2" [- testProperty "sort4" prop_sort4,- testProperty "sort5" prop_sort5,- testProperty "sort6" prop_sort6,- testCase "sort7" test_sort7,- testCase "sort8" test_sort8- ]- ]---prop_sort1 xs = sort xs == sortBy compare xs- where types = (xs :: [Int])--prop_sort2 xs =- (not (null xs)) ==>- (head (sort xs) == minimum xs)- where types = (xs :: [Int])--prop_sort3 xs = (not (null xs)) ==>- last (sort xs) == maximum xs- where types = (xs :: [Int])--prop_sort4 xs ys =- (not (null xs)) ==>- (not (null ys)) ==>- (head (sort (xs ++ ys)) == min (minimum xs) (minimum ys))- where types = (xs :: [Int], ys :: [Int])--prop_sort5 xs ys =- (not (null xs)) ==>- (not (null ys)) ==>- (head (sort (xs ++ ys)) == max (maximum xs) (maximum ys))- where types = (xs :: [Int], ys :: [Int])--prop_sort6 xs ys =- (not (null xs)) ==>- (not (null ys)) ==>- (last (sort (xs ++ ys)) == max (maximum xs) (maximum ys))- where types = (xs :: [Int], ys :: [Int])--test_sort7 = sort [8, 7, 2, 5, 4, 9, 6, 1, 0, 3] @?= [0..9]--test_sort8 = error "This test deliberately contains a user error"-\end{code}
− Test/Framework/Providers/HUnit.hs
@@ -1,56 +0,0 @@-module Test.Framework.Providers.HUnit (- testCase- ) where--import Test.Framework.Providers.API--import Test.HUnit.Lang----- | Create a 'Test' for a HUnit 'Assertion'-testCase :: TestName -> Assertion -> Test-testCase name = Test name . TestCase---instance TestResultlike TestCaseRunning TestCaseResult where- testSucceeded = testCaseSucceeded--data TestCaseRunning = TestCaseRunning--instance Show TestCaseRunning where- show TestCaseRunning = "Running"--data TestCaseResult = TestCasePassed- | TestCaseFailed String- | TestCaseError String--instance Show TestCaseResult where- show result = case result of- TestCasePassed -> "OK"- TestCaseFailed message -> "Failed: " ++ message- TestCaseError message -> "ERROR: " ++ message--testCaseSucceeded :: TestCaseResult -> Bool-testCaseSucceeded TestCasePassed = True-testCaseSucceeded _ = False---newtype TestCase = TestCase Assertion--instance Testlike TestCaseRunning TestCaseResult TestCase where- runTest topts (TestCase assertion) = runTestCase topts assertion- testTypeName _ = "Test Cases"--runTestCase :: CompleteTestOptions -> Assertion -> IO (TestCaseRunning :~> TestCaseResult, IO ())-runTestCase topts assertion = runImprovingIO $ do- yieldImprovement TestCaseRunning- mb_result <- maybeTimeoutImprovingIO (unK $ topt_timeout topts) $ liftIO (myPerformTestCase assertion)- return (mb_result `orElse` TestCaseError "Timed out")--myPerformTestCase :: Assertion -> IO TestCaseResult-myPerformTestCase assertion = do- result <- performTestCase assertion- return $ case result of- Nothing -> TestCasePassed- Just (True, message) -> TestCaseFailed message- Just (False, message) -> TestCaseError message
− Test/Framework/Providers/QuickCheck.hs
@@ -1,96 +0,0 @@-module Test.Framework.Providers.QuickCheck (- testProperty- ) where--import Test.Framework.Providers.API--import Test.QuickCheck hiding (Property)--import Data.List--import System.Random----- | Create a 'Test' for a QuickCheck 'Testable' property-testProperty :: Testable a => TestName -> a -> Test-testProperty name = Test name . Property---instance TestResultlike PropertyTestCount PropertyResult where- testSucceeded = propertySucceeded---- | Used to document numbers which we expect to be intermediate test counts from running properties-type PropertyTestCount = Int---- | The failure information from the run of a property-data PropertyResult = PropertyResult {- pr_status :: PropertyStatus,- pr_used_seed :: Int,- pr_tests_run :: Maybe PropertyTestCount -- Due to technical limitations, it's currently not possible to find out the number of- -- tests previously run if the test times out, hence we need a Maybe here for that case.- }--data PropertyStatus = PropertyOK -- ^ The property is true as far as we could check it- | PropertyArgumentsExhausted -- ^ The property may be true, but we ran out of arguments to try it out on- | PropertyFalsifiable [String] -- ^ The property was not true. The list of strings are the arguments inducing failure.- | PropertyTimedOut -- ^ The property timed out during execution--instance Show PropertyResult where- show (PropertyResult { pr_status = status, pr_used_seed = used_seed, pr_tests_run = mb_tests_run })- = case status of- PropertyOK -> "OK, passed " ++ tests_run_str ++ " tests"- PropertyArgumentsExhausted -> "Arguments exhausted after " ++ tests_run_str ++ " tests"- PropertyFalsifiable test_args -> "Falsifiable with seed " ++ show used_seed ++ ", after " ++ tests_run_str ++ " tests:\n" ++ unlinesConcise test_args- PropertyTimedOut -> "Timed out after " ++ tests_run_str ++ " tests"- where- tests_run_str = fmap show mb_tests_run `orElse` "an unknown number of"--propertySucceeded :: PropertyResult -> Bool-propertySucceeded result = propertyStatusIsSuccess (pr_status result)--propertyStatusIsSuccess :: PropertyStatus -> Bool-propertyStatusIsSuccess PropertyOK = True-propertyStatusIsSuccess PropertyArgumentsExhausted = True-propertyStatusIsSuccess _ = False---data Property = forall a. Testable a => Property a--instance Testlike PropertyTestCount PropertyResult Property where- runTest topts (Property testable) = runProperty topts testable- testTypeName _ = "Properties"--runProperty :: Testable a => CompleteTestOptions -> a -> IO (PropertyTestCount :~> PropertyResult, IO ())-runProperty topts testable = do- (gen, seed) <- newSeededStdGen (unK $ topt_seed topts)- runImprovingIO $ do- mb_result <- maybeTimeoutImprovingIO (unK (topt_timeout topts)) $ myCheck topts gen testable- return $ toPropertyResult seed $ case mb_result of- Nothing -> (PropertyTimedOut, Nothing)- Just (status, tests_run) -> (status, Just tests_run)- where- toPropertyResult seed (status, mb_tests_run) = PropertyResult {- pr_status = status,- pr_used_seed = seed,- pr_tests_run = mb_tests_run- }--myCheck :: (Testable a) => CompleteTestOptions -> StdGen -> a -> ImprovingIO PropertyTestCount f (PropertyStatus, PropertyTestCount)-myCheck topts rnd a = myTests topts (evaluate a) rnd 0 0 []--myTests :: CompleteTestOptions -> Gen Result -> StdGen -> PropertyTestCount -> PropertyTestCount -> [[String]] -> ImprovingIO PropertyTestCount f (PropertyStatus, PropertyTestCount)-myTests topts gen rnd0 ntest nfail stamps- | ntest == unK (topt_maximum_generated_tests topts) = do return (PropertyOK, ntest)- | nfail == unK (topt_maximum_unsuitable_generated_tests topts) = do return (PropertyArgumentsExhausted, ntest)- | otherwise = do- yieldImprovement ntest- case ok result of- Nothing ->- myTests topts gen rnd1 ntest (nfail + 1) stamps- Just True ->- myTests topts gen rnd1 (ntest + 1) nfail (stamp result:stamps)- Just False ->- return (PropertyFalsifiable (arguments result), ntest)- where- result = generate (configSize defaultConfig ntest) rnd2 gen- (rnd1, rnd2) = split rnd0
test-framework.cabal view
@@ -1,11 +1,11 @@ Name: test-framework-Version: 0.1.2+Version: 0.2.0 Cabal-Version: >= 1.2 Category: Testing Synopsis: Framework for running and organising tests, with HUnit and QuickCheck support-Description: Allows QuickCheck properties and HUnit test cases to be assembled into test groups, run in parallel (but reported- in deterministic order, to aid diff interpretation) and filtered and controlled by command line options.- All of this comes with colored test output, progress reporting and test statistics output.+Description: Allows tests such as QuickCheck properties and HUnit test cases to be assembled into test groups, run in+ parallel (but reported in deterministic order, to aid diff interpretation) and filtered and controlled by+ command line options. All of this comes with colored test output, progress reporting and test statistics output. License: BSD3 License-File: LICENSE Extra-Source-Files: README.textile@@ -22,19 +22,7 @@ Description: Build the tests Default: False -Flag Example- Description: Build the example testsuite- Default: False -Flag HUnit- Description: Enable HUnit integration: disabling this is not recommended, as it changes the package API- Default: True--Flag QuickCheck- Description: Enable QuickCheck integration: disabling this is not recommended, as it changes the package API- Default: True-- Library Exposed-Modules: Test.Framework Test.Framework.Options@@ -57,14 +45,6 @@ Test.Framework.Runners.TimedConsumption Test.Framework.Utilities - if flag(quickcheck)- Build-Depends: QuickCheck >= 1.1- Exposed-Modules: Test.Framework.Providers.QuickCheck- - if flag(hunit)- Build-Depends: HUnit >= 1.2- Exposed-Modules: Test.Framework.Providers.HUnit- Build-Depends: ansi-terminal >= 0.4.0, ansi-wl-pprint >= 0.4.0, regex-posix >= 0.72 if flag(splitBase) Build-Depends: base >= 3, random >= 1.0, containers >= 0.1@@ -89,7 +69,7 @@ Executable test-framework-tests Main-Is: Test/Framework/Tests.hs - Build-Depends: QuickCheck >= 1.1, HUnit >= 1.2, ansi-terminal >= 0.4.0, ansi-wl-pprint >= 0.4.0, regex-posix >= 0.72+ Build-Depends: HUnit >= 1.2, ansi-terminal >= 0.4.0, ansi-wl-pprint >= 0.4.0, regex-posix >= 0.72 if flag(splitBase) Build-Depends: base >= 3, random >= 1.0, containers >= 0.1 else@@ -111,31 +91,4 @@ Cpp-Options: -DCOMPILER_GHC if !flag(tests)- Buildable: False--Executable test-framework-example- Main-Is: Test/Framework/Example.lhs-- Build-Depends: QuickCheck >= 1.1, HUnit >= 1.2, ansi-terminal >= 0.4.0, ansi-wl-pprint >= 0.4.0, regex-posix >= 0.72- if flag(splitBase)- Build-Depends: base >= 3, random >= 1.0, containers >= 0.1- else- Build-Depends: base < 3- - Extensions: CPP- PatternGuards- ExistentialQuantification- RecursiveDo- FlexibleInstances- TypeSynonymInstances- TypeOperators- FunctionalDependencies- MultiParamTypeClasses- - Ghc-Options: -threaded- - if impl(ghc)- Cpp-Options: -DCOMPILER_GHC- - if !flag(example) Buildable: False