sydtest-0.27.2.0: CHANGELOG.md

# Changelog

## [0.27.2.0] - 2026-07-16

### Changed

* Mutation testing now runs each mutation's covering tests cheapest-first,
  using the per-test timings measured during the coverage phase.  Because a
  mutation child runs its covering tests under fail-fast, reaching a failing
  (killing) test sooner cuts the mutation phase's wall-clock; on two sample
  projects the mutation phase ran roughly 6% and 24% faster, with identical
  verdicts.  The ordering is a drop-in alternative to execution-order
  randomisation: it is applied only when the suite randomises execution order,
  and it never reorders a `doNotRandomiseExecutionOrder` block, so which tests
  run (and therefore every mutation verdict) is unchanged.

## [0.27.1.0] - 2026-06-29

### Fixed

* The spec-forest runners now seed the global `System.Random` generator from
  the configured seed before running.  Previously only the top-level `sydTest`
  runner did this, so the mutation and coverage children — which run a forest
  directly — left the global generator unseeded.  As a result, tests that draw
  from the global generator (`randomIO`, `newStdGen`, ...) rather than from
  QuickCheck's replay seed were non-reproducible under those children, which
  could produce spurious mutation kills or survivors.

## [0.27.0.0] - 2026-06-20

### Changed

* `renderMutationProgressEvent` now takes the mutation's 1-based index and the
  total mutation count as two `Int` arguments, and prefixes each per-mutation
  progress line with a coloured `[X/Y]` counter so a long mutation run shows
  how far along it is.

## [0.26.1.0] - 2026-06-17

### Added

* The mutation run report now prints, under each surviving mutation, the
  exact disable annotation for it (using the mutation's operator and enclosing
  binding) and any mitigation hint the operator attached to it.  Exposed as
  `survivorMitigationLines` from `Test.Syd.MutationMode`.

## [0.26.0.0] - 2026-06-09

### Changed

* `renderMutationProgressEvent` now takes a `Bool` (verbose) first argument.
  The mutation runner prints a concise one-line progress message per
  mutation by default and the full source diff of each mutation only when
  the new `--debug` driver flag is set, so a long run no longer floods the
  log with the diffs of killed mutations.

## [0.25.0.1] - 2026-06-04

### Changed

* Diff-rendering helpers (`renderUnifiedDiff` and the colour helpers
  `delColour`, `addColour`, `emphasiseIntraLine`, `renderDelSide`,
  `renderAddSide`) moved into `sydtest-mutation-runtime` so the plugin's
  manifest writer and `sydtest`'s mutation report share one
  implementation.  `Test.Syd.Output.Common` and
  `Test.Syd.MutationMode.Common` re-export them; existing call sites are
  unaffected.

## [0.25.0.0] - 2026-05-21

### Added

* `--mutation-coverage-list-locations`: a child-side mutation mode that prints,
  as a JSON array, each leaf test's id and the source location of its
  `it`/`prop`/`specify` call site.  Used by the diff-scoped mutation runner to
  map a changed test-source line back to the tests defined there.
* `flattenTestForestWithIdsAndCallStacks` in `Test.Syd.Mutation.Forest`, which
  pairs each leaf `TestId` with the `CallStack` captured at its definition site.

## [0.24.0.0] - 2026-05-18

### Added

* Mutation testing.  Instrument a library with the
  `sydtest-mutation-plugin` GHC plugin to record mutation sites, then run the
  test suite with `--mutation-coverage` and `--mutation` to find untested
  behaviour.  Surviving mutations indicate gaps in the test suite.
* `delColour`, `addColour`, `emphasiseIntraLine`, `renderDelSide`,
  `renderAddSide` in `Test.Syd.Output.Common` for diff rendering.

### Changed

* Equality-assertion diff output (`formatDiff`) now uses the same colour
  scheme as the mutation-mode unified diff: paired-line foregrounds are
  dull red/green, intra-line changes are bold + bright red/green, and
  whitespace-only changes get a background fill.

## [0.23.0.2] - 2026-02-26

### Changed

* Fixed a missing import on Windows.

## [0.23.0.1] - 2026-01-26

### Changed

* Fixed AI autodetection

## [0.23.0.0] - 2026-01-26

### Added

* Terse output format
* AI-executor detection

### Changed

* Simplified outputting code.

This is technically a breaking change, but if you are not using the sydtest
output code directly, it should not break anything for you.

## [0.22.0.0] - 2025-09-28

### Changed

* Fail on an empty test suite.

## [0.21.0.0] - 2025-05-09

This is technically a breaking change, but if you are not using the sydtest
constructors directly, it should not break anything for you.

### Added

* `aroundAllWithAll`
* `setupAroundAllWithAll`

### Changed

* Gave the 'DefAroundAllWithNode' constructor access to all outer resources
  instead of just the latest one.

## [0.20.0.1] - 2025-05-09

### Added

* Staged golden tests

## [0.20.0.0] - 2025-05-09

### Added

- `--skip-passed`: Skip passing tests and rerun once the entire suite is
  skipped.

## [0.19.0.0] - 2024-11-17

### Added

- Timeout support
- `modifiedTimeout`
- `withoutTimeout`
- `withTimeout`

### Changed

- Tests now timeout after 60 seconds by default.

## [0.18.0.1] - 2024-11-01

### Changed

- Fixed `mkNotEqualButShouldHaveBeenEqual` logic so it keeps the escape
  sequence for `Text` and `String`. This fix a regression introduced in
  0.18.0.0.

## [0.18.0.0] - 2024-09-26

### Added

- The test `Assertion` which displays a diff in case of error (so `shouldBe`,
  `shouldReturn`, golden tests and variations) will now timeout (after `2s`)
  when computing the diff between expected and actual value.
  In case of timeout, the values are displayed without any diff formatting.
  This ensure that test suite runtime won't be dominated by computing diff on
  some pathological cases.
- The smart constructor `mkNotEqualButShouldHaveBeenEqual` 
- You can use your own diff algorithm using the constructor
  `NotEqualButShouldHaveBeenEqualWithDiff`.
- Test suite does not crash if failed assertion tries to print values
  containing lazy exception.
  For example `shouldBe (1, error "nop") (2, 3)` was crashing before.
  The exception is now reported as the failure reason for the test.
  Note that this can be counter intuitive, because the test is failing because
  values are not equal (e.g. `(1, _) != (2, _)`), and this will be reported
  differently.


### Changed

The diff computation between actual value and reference changed so diff can
timeout.

This does not change the usual API (`shouldBe` or `GoldenTest`), but some
internal changed and you may need to adapt.
The change is straightforward, most of the functions are not `IO`:

- `stringsNotEqualButShouldHaveBeenEqual`,
  `textsNotEqualButShouldHaveBeenEqual` and
  `bytestringsNotEqualButShouldHaveBeenEqual` are now `IO Assertion` (was
  `Assertion`) in order to implement the timeout logic described for
  `shouldBe`.
  The `Assertion` `NotEqualButShouldHaveBeenEqual` is removed and replaced by
  `NotEqualButShouldHaveBeenEqualWithDiff` which embed the difference between
  both values.
- The record field `goldenTestCompare` of `GoldenTest` changed from `a -> a ->
  Maybe Assertion` to `a -> a -> IO (Maybe Assertion)`.

## [0.17.0.2] - 2024-09-26

### Changed

- Sydtest won't crash anymore, behave weirdly, or leak resources when executed
  in a REPL and interrupted by C-c.

## [0.17.0.1] - 2024-09-26

### Changed

* Only use `withArgs` when the argument list isn't already empty.
  This works around a concurrency issue wherein `withArgs` cannot be run twice from multiple threads.

## [0.17.0.0] - 2024-08-04

### Changed

* Allow golden tests to perform IO during comparisons

## [0.16.0.0] - 2024-08-03

### Changed

* `opt-env-conf`-based settings parsing.

## [0.15.1.3] - 2024-07-20

### Changed

* Fix race condition in the asynchronous runner

## [0.15.1.2] - 2024-07-18

### Changed

* Fix parsing filter flags so it becomes easy to select tests with spaces in their description

## [0.15.1.1] - 2023-10-04

### Changed

* Compatibility with `optparse-applicative > 0.18`.
* Compatibility with `GHC >= 9.7`.
* Refactored out `fast-myers-diff` into its own package.

## [0.15.1.0] - 2023-07-28

### Added

* `setupAroundWithAll`: so it's easier to use multiple outer resources to provide an inner resource, without the need of extra type annotation.

## [0.15.0.0] - 2023-04-08

### Added

* `DefBeforeAllWithNode`: so that `beforeAllWith` can be defined in terms of it and have better parallelism properties.
* `DefSetupNode`: so that `beforeAll_` can be defined in terms of it and have better parallelism properties.

## [0.14.0.0] - 2023-04-05

### Added

* Profiling mode, for figuring out why your test suite is slow.
  Use `--profile` to turn it on.

### Changed

* An improved asynchronous test runner.
* Made `--debug` imply `--retries 0`

## [0.13.0.4] - 2023-03-31

### Added

* Allow supplying multiple `--filter`/`--match` flags.
* Add short names `-m` and `-f` for the filter flag options.

## [0.13.0.3] - 2023-03-27

### Added

* Compatibility with `mtl-2.3`

## [0.13.0.2] - 2023-03-24

### Changed

* Improved white space diffing

## [0.13.0.1] - 2023-01-13

### Changed

* Replaced the diffing algorithm by a custom one.
  This has sped up diffing at least 100x and let us get rid of the `Diff` and `split` dependencies.
* Reworded 'likely not flaky' to 'does not look flaky' to be more technically
  accurate as we know nothing about the likelihood of flakiness.

## [0.13.0.0] - 2022-10-14

### Changed

* Changed the default of --golden-start to false.
  If this causes any of your tests to fail, that means that you are not
  correctly shipping the golden results along with your test suite.
* Fixed a typo in the parallelisation warning that you see when you run sydtest in parallel with only one core..

## [0.12.0.1] - 2022-10-06

* Released the changelog with a date.

## [0.12.0.0] - 2022-10-06

### Added

* Automatic flakiness diagnostics, see the `README`
  This adds the `potentiallyFlaky` and `potentiallyFlakyWith` functions.
* `Test.Syd` now also exports `pPrint`.
* The `modifyRetries`, `withoutRetries`, `withRetries` functions, to allow configuration of the number of retries independently of whether flakiness is allowed.
* The `TestRunReport` type, so that a `ResultForest` contains information about all runs of a test instead of only the last.
* Expectation of failure.
  This adds the `expectFailing`, `expectPassing`, and `withExpectationMode` functions.

### Changed

* Fixed: Fail-fast now works correctly together with fail-on-flaky
* Fixed that flags with a `no-` prefix did not parse correctly and could therefore not be used.
* The `FlakinessMode` type no longer contains a number of retries.
  The number of retries is now configured separately.
* Fixed that `xdescribe` would only result in one pending test instead of the same number of tests as are marked as pending.
* Fixed that `specify` and `prop` would show a callstack from inside `sydtest` instead of from where you used them.
* Sydtest now sets the global pseudorandomness seed in the random library to the given seed using `setStdGen` for extra reproducability.

### Removed

## [0.11.0.2] - 2022-09-7

### Changed

* Added the `shouldMatchList` expectation, corresponding the one from
  `hspec-expectations`.

## [0.11.0.1] - 2022-06-28

### Changed

* Made the type of `setupAround` more general.

## [0.11.0.0] - 2022-06-28

### Changed

* `printOutputSpecForest` now uses `Data.Text.Lazy.IO` to output the report, so it will try to respect the system's locale.
* `renderResultReport` now outputs a `Text.Builder` instead of a `ByteString.Builder` and a newline at the end.

## [0.10.1.0] - 2022-06-04

### Changed

* Fixed a bug where multiple layers of contextual failures were not unwrapped and shown correctly.
* Made 'Assertion' fields strict.

## [0.10.0.0] - 2022-04-28

### Added

* Allow supplying filters without a flag, so `hello` instead of `--filter hello` or `--match hello`.
* Simplified the way exceptions work to allow for attaching context to any failure instead of only `Assertion`s.

## [0.9.0.0] - 2022-04-13

### Added

* Optional type-safe progress reporting for property tests.

## [0.8.0.1] - 2022-02-11

### Changed

* Fixed property tests without labels still showing labels in the output.

## [0.8.0.0] - 2022-02-11

### Changed

* The `TestDefM` now contains a `TestDefEnv` which also contains the test description path, along with `TestRunSettings`.
* Removed the `MonadState ()` instance of `TestDefM`. It was just silly.
* Changed the internals of `TestDefM` to use `ReaderT` and a strict `WriterT` instead of `RWST`.
* Renamed `wrapRWST` to `wrapForest`.
* Fixed the property label output to use the right total.
* Moved `filterTestForest` and `randomiseTestForest` from `Test.Syd.TestDefM` to `Test.Syd.SpecDef`, where `SpecDefForest` is defined.

### Added

* `getTestDescriptionPath` to get the test description path upwards from inside a test definition.


## [0.7.0.1] - 2021-12-23

### Changed

* Fixed `shouldStartWith` to test on the prefix rather then infix

## [0.7.0.0] - 2021-12-15

### Added

* Flaky tests now show up in the failure report when `--fail-on-flaky` is active.
* Flakiness information like the number of retries is now shown in the failure report for real (non-flaky) failures.

### Changed

* Simplified the way settings are passed around.

## [0.6.1.0] - 2021-12-10

### Added

* The `flakyWith` combinator, which is like `flaky`, but lets you also add a message to your team.

## [0.6.0.0] - 2021-11-12

### Changed

* Started using `autodocodec` instead of `yamlparse-applicative`.

## [0.5.0.0] - 2021-11-12

### Added

* The flakiness combinators (`flaky`, `notFlaky`, and `withFlakiness`) to mark a test group as potentially flaky.
* The `--fail-on-flaky` flag to falsify flakiness.
* Experimental Windows support

### Changed

* Fixed the interpretation of `max-size` vs `max-success` in the configuration file and environment parsing.

## [0.4.1.0] - 2021-10-10

### Added

* The `--random-seed` option to use random seeds instead of the fixed seed that is used by default.

## [0.4.0.0] - 2021-09-02

### Added

* The `--debug` option.

### Changed

* Redid the entire flags parsing.
  This should be backward compatible, and result in a nicer `--help` overview.

## [0.3.0.3] - 2021-08-07

### Changed

* Show the total number of examples in the output as well

## [0.3.0.2] - 2021-07-06

### Changed

* Accept options using American spelling as well.

## [0.3.0.1] - 2021-06-20

### Changed

* Turned off shrinking when using `around` and friends. See https://github.com/nick8325/quickcheck/issues/331.

## [0.3.0.0] - 2021-06-17

### Added

* An `IsTest (ReaderT env IO a)` instance.

### Deleted

* `Test.Syd.Def.Env`, which contained `eit` and `withTestEnv`
  Now that `ReaderT env IO a` is also in `IsTest`, you can just use `it` for this.

## [0.2.0.0] - 2021-06-03

### Added

* `beforeWith` and `beforeWith'`
* `scenarioDir` and `scenarioDirRecur` for scenario testing.
* `bracketSetupFunc`

### Changed

* The `SetupFunc` has been simplified to only take one type parameter.

### Deleted

* `composeSetupFunc`, now obsolete: use `<=<` instead.
* `connectSetupFunc`, now obsolete: use `>=>` instead.
* `wrapSetupFunc`, now entirely obsolete.
* `unwrapSetupFunc`, now entirely obsolete.
* `makeSimpleSetupFunc`, now obsolete: Use the `SetupFunc` constructor directly.
* `useSimpleSetupFunc`, now obsolete: Use the `unSetupFunc` function directly.

## [0.1.0.0] - 2021-03-07

Various fixes

## [0.0.0.0] - 2020-12-26

Initial release