golds-gym 0.4.0.0 → 0.5.0.0

raw patch · 6 files changed

+32/−80 lines, 6 filesPVP ok

version bump matches the API change (PVP)

API changes (from Hackage documentation)

Files

• CHANGELOG.md +14/−3
• README.md +4/−11
• example/Spec.hs +1/−1
• golds-gym.cabal +1/−1
• src/Test/Hspec/BenchGolden.hs +4/−15
• src/Test/Hspec/BenchGolden/Runner.hs +8/−49

@@ -1,5 +1,19 @@ # Changelog +## [0.5.0]++### Added++- Clarified copyright notice for vendored code from `tasty-bench`, (c) Andrew Lelechenko 2021 -++### Removed++- **BREAKING:** Removed `whnf` (weak head normal form) evaluation combinators, the library is geared towards non-infinite input data+  - Removed `whnf` - use `nf` instead for pure functions+  - Removed `whnfIO` - use `nfIO` instead for IO actions+  - Removed `whnfAppIO` - use `nfAppIO` instead for functions returning IO+  - Migration: Replace all `whnf*` calls with their `nf*` equivalents+ ## [0.4.0]  ### Added@@ -14,11 +28,8 @@  - **Evaluation strategy combinators** for proper laziness handling   - `nf` - Force result to normal form (deep evaluation)-  - `whnf` - Force result to weak head normal form (shallow evaluation)   - `nfIO` - Normal form for IO actions-  - `whnfIO` - Weak head normal form for IO actions   - `nfAppIO` - Normal form for functions returning IO-  - `whnfAppIO` - WHNF for functions returning IO   - `io` - Plain IO action (for backward compatibility)   - Vendored evaluation loops from tasty-bench with attribution 

@@ -20,35 +20,28 @@ main :: IO () main = hspec $ do   describe "Performance" $ do-    -- Pure function with normal form evaluation (deep, full evaluation)+    -- Pure function with normal form evaluation (full evaluation of the function arguments)     benchGolden "list append" $       nf (\xs -> xs ++ xs) [1..1000] -    -- Weak head normal form (shallow, outermost constructor only)-    benchGolden "replicate" $-      whnf (replicate 1000) 42-     -- Custom configuration     benchGoldenWith defaultBenchConfig       { iterations = 500       , tolerancePercent = 10.0       }       "sorting" $-      nf sort [1000, 999..1]+      nf ...  ```  **Evaluation strategies** (required - specify how values are forced): - `nf f x` - Force result of `f x` to **normal form** (deep, full evaluation)-- `whnf f x` - Force result of `f x` to **weak head normal form** (shallow, outermost constructor only) - `nfIO action` - Execute IO action and force result to normal form-- `whnfIO action` - Execute IO action and force result to WHNF - `nfAppIO f x` - Apply function, execute resulting IO, force result to normal form-- `whnfAppIO f x` - Apply function, execute resulting IO, force result to WHNF - `io action` - Plain IO action without additional forcing -**Why evaluation strategies matter**: Without forcing, GHC may optimize away computations or share results across iterations, making benchmarks meaningless. Use `nf` for most cases unless you specifically want lazy evaluation (`whnf`).+**Why evaluation strategies matter**: Without forcing, GHC may optimize away computations or share results across iterations, making benchmarks meaningless. -**First run** creates `.golden/<arch>/list-append.golden` with baseline stats.  +**First run** creates `.golden/<arch>/<benchmark_name>.golden` with baseline stats.   **Subsequent runs** compare against baseline. Test fails if mean time changes beyond tolerance (default: ±15% OR ±0.01ms).  **Output format** :

@@ -132,7 +132,7 @@     -- Hybrid tolerance with custom absolute threshold     benchGoldenWithExpectation "hybrid tolerance custom"        defaultBenchConfig { iterations = 1000 }-      [expect _statsMean (Hybrid 5.0 0.0001)] +      [expect _statsMean (Hybrid 5.0 0.001)]        $ nf (\n -> sort [n, n-1 .. 1]) (200 :: Int)      -- Use robust statistics lens (trimmed mean)

@@ -1,6 +1,6 @@ cabal-version:      3.0 name:               golds-gym-version:            0.4.0.0+version:            0.5.0.0 synopsis:           Golden testing framework for performance benchmarks description:     A Haskell framework for golden testing of timing benchmarks.

@@ -33,11 +33,7 @@ --   describe \"Performance\" $ do --     -- Pure function with normal form evaluation --     `benchGolden` "list sorting" $---       `nf` sort [1000, 999..1]------     -- Weak head normal form (lazy evaluation)---     `benchGolden` "replicate" $---       `whnf` (replicate 1000) 42+--       `nf` (\\n -> sort [n, n-1 .. 1]) 1000 -- --     -- IO action with result forced to normal form --     `benchGolden` "file read" $@@ -47,9 +43,8 @@ -- __Evaluation strategies__ control how values are forced: -- -- * 'nf' - Force to normal form (deep evaluation, use for most cases)--- * 'whnf' - Force to weak head normal form (only outermost constructor is evaluated)--- * 'nfIO', 'whnfIO' - Variants for IO actions--- * 'nfAppIO', 'whnfAppIO' - For functions returning IO+-- * 'nfIO' - Variant for IO actions+-- * 'nfAppIO' - For functions returning IO -- * 'io' - Plain IO action without forcing -- -- Without proper evaluation strategies, GHC may optimize away computations@@ -197,11 +192,8 @@      -- * Benchmarkable Constructors   , nf-  , whnf   , nfIO-  , whnfIO   , nfAppIO-  , whnfAppIO   , io      -- * Low-Level API@@ -226,7 +218,7 @@ import Test.Hspec.BenchGolden.Arch import qualified Test.Hspec.BenchGolden.Lenses as L import Test.Hspec.BenchGolden.Lenses hiding (Expectation)-import Test.Hspec.BenchGolden.Runner (runBenchGolden, setAcceptGoldens, setSkipBenchmarks, nf, whnf, nfIO, whnfIO, nfAppIO, whnfAppIO, io)+import Test.Hspec.BenchGolden.Runner (runBenchGolden, setAcceptGoldens, setSkipBenchmarks, nf, nfIO, nfAppIO, io) import Test.Hspec.BenchGolden.Types  -- | Create a benchmark golden test with default configuration.@@ -242,11 +234,8 @@ -- Use evaluation strategy combinators to control how values are forced: -- -- * 'nf' - Normal form (deep evaluation)--- * 'whnf' - Weak head normal form (shallow evaluation) -- * 'nfIO' - Normal form for IO actions--- * 'whnfIO' - WHNF for IO actions -- * 'nfAppIO' - Normal form for functions returning IO--- * 'whnfAppIO' - WHNF for functions returning IO -- * 'io' - Plain IO action (for backward compatibility) -- -- Default configuration:

@@ -7,7 +7,7 @@ -- | -- Module      : Test.Hspec.BenchGolden.Runner -- Description : Benchmark execution and golden file comparison--- Copyright   : (c) 2026+-- Copyright   : (c) Marco Zocca 2026 -- License     : MIT -- Maintainer  : @ocramz --@@ -15,10 +15,10 @@ -- golden files. It includes: -- -- * Benchmark execution with warm-up iterations--- * Golden file I/O (reading/writing JSON statistics)+-- * Golden file IO (reading and writing JSON statistics) -- * Tolerance-based comparison with variance warnings -- * Support for updating baselines via GOLDS_GYM_ACCEPT environment variable--- * Evaluation strategies to control how values are forced (nf, whnf, etc.)+-- * Evaluation strategies to control how values are forced (nf variants). -- -- = Evaluation Strategies --@@ -26,14 +26,11 @@ -- optimizing away computations or sharing results across iterations: -- -- * 'nf' - Force result to normal form (deep, full evaluation)--- * 'whnf' - Force result to weak head normal form (shallow evaluation) -- * 'nfIO' - Execute IO and force result to normal form--- * 'whnfIO' - Execute IO and force result to WHNF -- * 'nfAppIO' - Apply function, execute IO, force result to normal form--- * 'whnfAppIO' - Apply function, execute IO, force result to WHNF -- * 'io' - Plain IO without additional forcing ----- These are vendored from tasty-bench with proper attribution (BSD-3-Clause).+-- These are vendored from tasty-bench under the MIT license, (c) 2021 Andrew Lelechenko.  module Test.Hspec.BenchGolden.Runner   ( -- * Running Benchmarks@@ -68,11 +65,8 @@     -- * Benchmarkable Constructors   , io   , nf-  , whnf   , nfIO-  , whnfIO   , nfAppIO-  , whnfAppIO   ) where  import Control.DeepSeq (NFData, rnf)@@ -100,19 +94,12 @@ import Test.Hspec.BenchGolden.Lenses (metricFor, varianceFor) import Test.Hspec.BenchGolden.Types --- -------------------------------------------------------------------------------- Evaluation Strategies--- Vendored from tasty-bench-0.5 with modifications--- Copyright (c) 2021 Andrew Lelechenko and tasty-bench contributors--- MIT License--- https://hackage.haskell.org/package/tasty-bench-0.5--- ------------------------------------------------------------------------------ -- | Benchmark a pure function applied to an argument, forcing the result to -- normal form (NF) using 'rnf' from "Control.DeepSeq". -- This ensures the entire result structure is evaluated. -- -- Example:+-- -- @ -- benchGolden "fib 30" (nf fib 30) -- @@@ -120,20 +107,10 @@ nf = funcToBench rnf {-# INLINE nf #-} --- | Benchmark a pure function applied to an argument, forcing the result to--- weak head normal form (WHNF) only. This evaluates just the outermost constructor.------ Example:--- @--- benchGolden "replicate" (whnf (replicate 1000) 42)--- @-whnf :: (a -> b) -> a -> BenchAction-whnf = funcToBench id-{-# INLINE whnf #-}- -- | Benchmark an 'IO' action, forcing the result to normal form. -- -- Example:+-- -- @ -- benchGolden "readFile" (nfIO $ readFile "data.txt") -- @@@ -141,19 +118,10 @@ nfIO = ioToBench rnf {-# INLINE nfIO #-} --- | Benchmark an 'IO' action, forcing the result to weak head normal form.------ Example:--- @--- benchGolden "getLine" (whnfIO getLine)--- @-whnfIO :: IO a -> BenchAction-whnfIO = ioToBench id-{-# INLINE whnfIO #-}- -- | Benchmark a function that performs 'IO', forcing the result to normal form. -- -- Example:+-- -- @ -- benchGolden "lookup in map" (nfAppIO lookupInDB "key") -- @@@ -161,20 +129,11 @@ nfAppIO = ioFuncToBench rnf {-# INLINE nfAppIO #-} --- | Benchmark a function that performs 'IO', forcing the result to weak head normal form.------ Example:--- @--- benchGolden "query database" (whnfAppIO queryDB params)--- @-whnfAppIO :: (a -> IO b) -> a -> BenchAction-whnfAppIO = ioFuncToBench id-{-# INLINE whnfAppIO #-}- -- | Benchmark an 'IO' action, discarding the result. -- This is for backward compatibility with code that uses @IO ()@ actions. -- -- Example:+--  -- @ -- benchGolden "compute" (io $ do --   result <- heavyComputation