sydtest 0.32.0.0 → 0.33.0.0

raw patch · 3 files changed

+36/−20 lines, 3 files

Files

• CHANGELOG.md +21/−0
• src/Test/Syd/MutationMode/Common.hs +14/−19
• sydtest.cabal +1/−1

@@ -1,5 +1,26 @@ # Changelog +## [0.33.0.0] - 2026-10-01++### Fixed++* A timed-out mutation counts as killed again.++  Mutating code easily stops it terminating.  A `ConstBool` on a loop's exit+  condition turns `while done` into `while True`, and the suite then runs+  until its budget is gone rather than failing an assertion.  A suite that+  never finishes is still a suite that noticed the mutation, which is what a+  kill means.  0.32.0.0 scored those as neither killed nor survived and+  failed the run over them, so a mutation the tests had caught was reported+  as one they had not.++  The reason to doubt the kill is that a busy machine overruns the budget+  too, and crediting that would let load raise the score.  The retry, also+  new in 0.32.0.0, is what answers it: an overrun is given three goes, so one+  that survives all three is the mutation's doing rather than the machine's.+  The two belong together, and only the retry was worth keeping on its own.++ ## [0.32.0.0] - 2026-10-01  ### Changed

@@ -92,9 +92,11 @@   = MutationUncovered UncoveredMutation   | MutationKilled AugmentedMutationRecord   | -- | At least one suite's child exceeded its monotonic-clock timeout, and-    -- still did when given another go.  Counted as neither killed nor-    -- survived, because the tests caught nothing, but it fails the run all-    -- the same: see 'isMutationFailure'.+    -- still did when given another go.  Counted as killed in the overall+    -- score, and also recorded separately in the report: a mutation that+    -- turns a loop into one that never ends is one the suite noticed, which+    -- is what a kill means.  The retry is what makes that safe to say, since+    -- a single overrun on a busy machine would not have meant it.     MutationTimedOut TimedOutMutation   | MutationSurvived SurvivedMutation   | -- | The mutation was not tested because an earlier mutation in the same@@ -205,22 +207,17 @@ instance Exception CoverageFailFast  -- | A mutation result that should trip within-group fail-fast: the test--- suite did not detect the mutation (survivor), no test reaches the--- mutation site (uncovered), or the suite never finished (timed out).--- 'MutationSkipped' is itself a consequence of a prior failure and does not--- trip again.+-- suite did not detect the mutation (survivor) or no test reaches the+-- mutation site (uncovered).  Timeouts count as killed, so they do not+-- trip; 'MutationSkipped' is itself a consequence of a prior failure and+-- does not trip again. isMutationFailure :: MutationResult -> Bool isMutationFailure = \case   MutationSurvived _ -> True   MutationUncovered _ -> True   MutationKilled _ -> False-  -- A mutation easily turns a loop into one that never ends, so a run that-  -- keeps overrunning is the expected shape of a mutation nothing asserted-  -- against.  Treating it as its own quiet category would let those go-  -- unmeasured in bulk, so it fails like a survivor; a mutation that is-  -- meant to hang gets a disable annotation, as any other unwanted mutation-  -- does.-  MutationTimedOut _ -> True+  -- [ref:TimeoutIsAKill]+  MutationTimedOut _ -> False   MutationSkipped _ -> False   -- A passing control is the expected outcome, so it is not a failure (and must   -- not trip fail-fast).  A failed control IS a failure: a no-op cannot be@@ -281,10 +278,8 @@   where     step = \case       OutcomeKilled _ -> \t -> t {tallyKilled = tallyKilled t + 1}-      -- Not a kill. The tests did not catch the mutation; the run simply did-      -- not finish, which is a fact about the budget and the machine. Scoring-      -- it as a kill let a loaded machine raise the score.-      OutcomeTimedOut _ -> \t -> t {tallyTimedOut = tallyTimedOut t + 1}+      -- [ref:TimeoutIsAKill]+      OutcomeTimedOut _ -> \t -> t {tallyKilled = tallyKilled t + 1, tallyTimedOut = tallyTimedOut t + 1}       OutcomeSurvived _ -> \t -> t {tallySurvived = tallySurvived t + 1}       OutcomeUncovered _ -> \t -> t {tallyUncovered = tallyUncovered t + 1}       OutcomeSkipped _ -> \t -> t {tallySkipped = tallySkipped t + 1}@@ -336,7 +331,7 @@   let MutationTally {..} = mutationRunReportMutations       ControlTally {..} = mutationRunReportControls    in [ [chunk "Killed: ", fore green (chunk (T.pack (show mutationTallyKilled)))],-        [chunk "Timed out: ", fore yellow (chunk (T.pack (show mutationTallyTimedOut)))],+        [chunk "  (of which timed out: ", fore yellow (chunk (T.pack (show mutationTallyTimedOut))), chunk ")"],         [chunk "Survived: ", fore red (chunk (T.pack (show mutationTallySurvived)))],         [chunk "Uncovered: ", fore yellow (chunk (T.pack (show mutationTallyUncovered)))],         [chunk "Skipped: ", fore yellow (chunk (T.pack (show mutationTallySkipped)))]

@@ -5,7 +5,7 @@ -- see: https://github.com/sol/hpack  name:           sydtest-version:        0.32.0.0+version:        0.33.0.0 synopsis:       A modern testing framework for Haskell with good defaults and advanced testing features. description:    A modern testing framework for Haskell with good defaults and advanced testing features. Sydtest aims to make the common easy and the hard possible. See https://github.com/NorfairKing/sydtest#readme for more information. category:       Testing