kudzu (empty) → 0.1.0.0

raw patch · 7 files changed

+272/−0 lines, 7 filesdep +QuickCheckdep +basedep +hedgehog

Dependencies added: QuickCheck, base, hedgehog, hpc, kudzu, leancheck

Files

• CHANGELOG.md +6/−0
• LICENSE +30/−0
• README.md +38/−0
• app/Main.hs +5/−0
• kudzu.cabal +117/−0
• src/Kudzu.hs +72/−0
• test/KudzuTest.hs +4/−0

@@ -0,0 +1,6 @@+# Revision history for kudzu++## 0.1.0.0 -- 2022-07-01++* First version. Released on an unsuspecting world. Good Luck World.+  Kinda does the thing, yay?

@@ -0,0 +1,30 @@+Copyright (c) 2022, Shae Erisson++All rights reserved.++Redistribution and use in source and binary forms, with or without+modification, are permitted provided that the following conditions are met:++    * Redistributions of source code must retain the above copyright+      notice, this list of conditions and the following disclaimer.++    * Redistributions in binary form must reproduce the above+      copyright notice, this list of conditions and the following+      disclaimer in the documentation and/or other materials provided+      with the distribution.++    * Neither the name of Shae Erisson nor the names of other+      contributors may be used to endorse or promote products derived+      from this software without specific prior written permission.++THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

@@ -0,0 +1,38 @@+# KUDZU will slowly grow to cover all of your code++Kudzu is a library that throws test cases at your property tests until the code coverage no longer increases.++# WHY?++Property testing has no feedback loop, you randomly choose a number of test cases and hope for the best.++How do you know if your property tests were any good? The best feedback I know is to use [hpc](https://wiki.haskell.org/Haskell_program_coverage) and look at the pretty colored HTML output to see what code was exercised.++But wait, why do *I* have to look at the output? Isn't that why we have computers?++# HOW?++In Haskell, you can get [code coverage results](https://hackage.haskell.org/package/hpc/docs/Trace-Hpc-Reflect.html#v:examineTix) while your program is running!++# WHAT FEEDBACK LOOP?++The simplest feedback loop is to keep running random tests until new code coverage stops increasing.++# HOW DO I MAKE IT GO?++add kudzu to your test-suite depends,++# TELL ME MORE++The best write up of this idea is [Random Test Generation, Coverage Based](https://danluu.com/testing/).++# TODO++- [x] support HedgeHog+- [x] support QuickCheck+- [x] support LeanCheck+- [ ] figure out how to use Kudzu on Kudzu without looping forever++# EXAMPLE++You can see kudzu in use in the [tests](https://github.com/shapr/takedouble/blob/main/test/Main.hs) for [takedouble](https://github.com/shapr/takedouble/)

@@ -0,0 +1,5 @@+module Main where++main :: IO ()+main = do+  print "Hello World, Kudzu will cover your code?"

@@ -0,0 +1,117 @@+cabal-version:      3.0++-- Initial package description 'kudzu.cabal' generated by+-- 'cabal init'. For further documentation, see:+--   http://haskell.org/cabal/users-guide/+--+-- The name of the package.+name:               kudzu++-- The package version.+-- See the Haskell package versioning policy (PVP) for standards+-- guiding when and how versions should be incremented.+-- https://pvp.haskell.org+-- PVP summary:      +-+------- breaking API changes+--                   | | +----- non-breaking API additions+--                   | | | +--- code changes with no API change+version:            0.1.0.0++-- A short (one-line) description of the package.+synopsis:           coverage driven random testing framework++-- A longer description of the package.+-- description:++-- URL for the project homepage or repository.+homepage:           https://github.com/shapr/kudzu++-- A URL where users can report bugs.+-- bug-reports:++-- The license under which the package is released.+license:            BSD-3-Clause++-- The file containing the license text.+license-file:       LICENSE++-- The package author(s).+author:             Shae Erisson++-- An email address to which users can send suggestions, bug reports, and patches.+maintainer:         shae@scannedinavian.com++-- A copyright notice.+-- copyright:+category:           Testing++-- Extra files to be distributed with the package, such as examples or a README.+extra-source-files: CHANGELOG.md+                    README.md+                    LICENSE+description: Kudzu is a coverage driven random testing framework+source-repository head+    type: git+    location: https://github.com/shapr/kudzu.git+++library+    -- Modules exported by the library.+    exposed-modules:  Kudzu++    -- Modules included in this library but not exported.+    -- other-modules:++    -- LANGUAGE extensions used by modules in this package.+    -- other-extensions:++    -- Other library packages from which modules are imported.+    build-depends:    base+                    , QuickCheck+                    , hedgehog+                    , hpc+                    , leancheck+++    -- Directories containing source files.+    hs-source-dirs:   src++    -- Base language which the package is written in.+    default-language: Haskell2010+    ghc-options: -Wall++executable kudzu+    -- .hs or .lhs file containing the Main module.+    main-is:          Main.hs++    -- Modules included in this executable, other than Main.+    -- other-modules:++    -- LANGUAGE extensions used by modules in this package.+    -- other-extensions:++    -- Other library packages from which modules are imported.+    build-depends:+        base >=4.14 && < 4.17,+        kudzu++    -- Directories containing source files.+    hs-source-dirs:   app++    -- Base language which the package is written in.+    default-language: Haskell2010++test-suite kudzu-test+    -- Base language which the package is written in.+    default-language: Haskell2010++    -- The interface type and version of the test suite.+    type:             exitcode-stdio-1.0++    -- Directories containing source files.+    hs-source-dirs:   test++    -- The entrypoint to the test suite.+    main-is:          KudzuTest.hs++    -- Test dependencies.+    build-depends:    base >=4.14

@@ -0,0 +1,72 @@+{-# LANGUAGE OverloadedStrings #-}++module Kudzu where++import Control.Monad (unless)+import qualified Hedgehog as HH+import qualified Test.LeanCheck as LC+import qualified Test.QuickCheck as QC+import Trace.Hpc.Reflect (examineTix)+import Trace.Hpc.Tix (Tix (..), TixModule (..))++testUntilSameQCMany :: (Traversable t, QC.Testable a) => Int -> t a -> IO (t (Int, Maybe Integer))+testUntilSameQCMany howMany ts = do+  mapM (testUntilSameQC howMany) ts++-- | QuickCheck+testUntilSameQC :: QC.Testable a => Int -> a -> IO (Int, Maybe Integer)+testUntilSameQC n testable = do+  let rs = examineAndCount' <$> repeat testable+  r1 <- head rs+  grabUntilNSame 0 n n (tail rs) r1++examineAndCount' :: QC.Testable prop => prop -> IO Integer+examineAndCount' v = do+  -- poor QC, you got problems+  -- quickCheckWith (stdArgs {maxSize = 400, maxSuccess = 1}) v+  QC.quickCheck (QC.withMaxSuccess 1 v)+  tixModuleCount <$> examineTix++-- | Hedgehog+testUntilSameHHMany :: Traversable t => Int -> t HH.Property -> IO (t (Int, Maybe Integer))+testUntilSameHHMany howMany ps = do+  mapM (testUntilSameHH howMany) ps++testUntilSameHH :: Int -> HH.Property -> IO (Int, Maybe Integer)+testUntilSameHH n prop = do+  let rs = examineAndCountHH <$> repeat prop+  r1 <- head rs+  grabUntilNSame 0 n n (tail rs) r1++examineAndCountHH :: HH.Property -> IO Integer+examineAndCountHH prop = do+  passed <- HH.check prop+  unless passed $ error "property failed"+  tixModuleCount <$> examineTix++-- | LeanCheck+testUntilSameLC :: LC.Testable a => Int -> a -> IO (Int, Maybe Integer)+testUntilSameLC n testable = do+  let rs = examineAndCount <$> LC.results testable+  r1 <- head rs+  grabUntilNSame 0 n n (tail rs) r1++examineAndCount :: ([String], Bool) -> IO Integer+examineAndCount v = unless (snd v) (error "your code is broken") >> tixModuleCount <$> examineTix++grabUntilNSame :: (Monad m, Eq a) => Int -> Int -> Int -> [m a] -> a -> m (Int, Maybe a)+grabUntilNSame c _ 0 _ z = pure (c, Just z)+grabUntilNSame c _ _ [] _ = pure (c, Nothing)+grabUntilNSame c orig n (a : as) z = do+  a' <- a+  if a' == z+    then grabUntilNSame (c + 1) orig (n - 1) as z+    else grabUntilNSame (c + 1) orig orig as a'++-- How many regions were executed at least once for this module?+tixCount :: TixModule -> Integer+tixCount (TixModule _ _ _ regions) = sum $ 1 <$ filter (> 0) regions++-- How many regions were executed at least once for all these modules?+tixModuleCount :: Tix -> Integer+tixModuleCount (Tix ms) = sum $ map tixCount ms

@@ -0,0 +1,4 @@+module Main (main) where++main :: IO ()+main = putStrLn "Test suite not yet implemented."