hpc-threshold (empty) → 0.1.0.0

raw patch · 8 files changed

+280/−0 lines, 8 filesdep +basedep +bytestringdep +hpc-thresholdsetup-changed

Dependencies added: base, bytestring, hpc-threshold, interpolate, pcre-heavy

Files

• ChangeLog.md +3/−0
• LICENSE +30/−0
• README.md +98/−0
• Setup.hs +2/−0
• app/Main.hs +6/−0
• hpc-threshold.cabal +70/−0
• src/Lib.hs +69/−0
• test/Spec.hs +2/−0

@@ -0,0 +1,3 @@+# Changelog for hpc-threshold++## Unreleased changes

@@ -0,0 +1,30 @@+Copyright Author name here (c) 2018++All rights reserved.++Redistribution and use in source and binary forms, with or without+modification, are permitted provided that the following conditions are met:++    * Redistributions of source code must retain the above copyright+      notice, this list of conditions and the following disclaimer.++    * Redistributions in binary form must reproduce the above+      copyright notice, this list of conditions and the following+      disclaimer in the documentation and/or other materials provided+      with the distribution.++    * Neither the name of Author name here nor the names of other+      contributors may be used to endorse or promote products derived+      from this software without specific prior written permission.++THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

@@ -0,0 +1,98 @@+# hpc-threshold++This is a small utility for validating whether HPC result is above some defined thresholds. This program is meant to be used within a CI pipeline, in which the build will fail if the code coverage falls below the thresholds.++The program reads a configuration file named `.hspec-threshold` and parse HPC text from stdin. The program outputs a report and will terminate with status code 1 if the coverage falls below threshold, and 0 otherwise.++## User Guide++Install the utility by using stack++```+stack install hpc-threshold+```++Then, create a configuration file named `.hspec-threshold`:++```+[ Threshold +    { thresholdName = "Expressions used"+    , thresholdRegex = "(\\d+)% expressions used"+    , thresholdValue = 80.0+    }+, Threshold +    { thresholdName = "Boolean coverage"+    , thresholdRegex = "(\\d+)% boolean coverage"+    , thresholdValue = 80.0+    }+, Threshold +    { thresholdName = "Alternatives used"+    , thresholdRegex = "(\\d+)% alternatives used"+    , thresholdValue = 80.0+    }+, Threshold +    { thresholdName = "Local declarations used"+    , thresholdRegex = "(\\d+)% local declarations used"+    , thresholdValue = 80.0+    }+, Threshold +    { thresholdName = "Top-level declarations used"+    , thresholdRegex = "(\\d+)% top-level declarations used"+    , thresholdValue = 80.0+    }+]+```++- `thresholdRegex` is the regex to be used for extracting the coverage from HPC report. It requires 1 digit capture.+- `thresholdValue` is the threshold for the code coverage.+- `thresholdName` will be used for the threshold report++Then, build the coverage report:++```+stack test --coverage+```++Then, generate a text report and feed that into `hpc-threshold`:++```+(stack hpc report --all 2&>1) | hpc-threshold+```++The stderr -> stdout redirection is necessary there because `stack hpc report` outputs the result in stderr, but we want to pipe that into `hpc-threshold`.++Then, you'll get an output similar to the following:++```+Code coverage threshold check: FAIL+· Expressions used: 67.0% (< 80.0%)+· Boolean coverage: 14.0% (< 80.0%)+· Alternatives used: 42.0% (< 80.0%)+✓ Local declarations used: 88.0% (≥ 80.0%)+✓ Top-level declarations used: 80.0% (≥ 80.0%)+```++If we check the exit code of the last process, we'll get `1` since some coverage areas are below the configured threshold++```+$ echo $?+1+```++For successful scenario, the output that you'll get is as follows:++```+Code coverage threshold check: PASS+✓ Expressions used: 67.0% (≥ 60.0%)+✓ Boolean coverage: 14.0% (≥ 10.0%)+✓ Alternatives used: 42.0% (≥ 40.0%)+✓ Local declarations used: 88.0% (≥ 80.0%)+✓ Top-level declarations used: 80.0% (≥ 80.0%)+```++And the exit code is 0++```+$ echo $?+0+```

@@ -0,0 +1,2 @@+import Distribution.Simple+main = defaultMain

@@ -0,0 +1,6 @@+module Main where++import Lib++main :: IO ()+main = evaluateAndReport'

@@ -0,0 +1,70 @@+-- This file has been generated from package.yaml by hpack version 0.20.0.+--+-- see: https://github.com/sol/hpack+--+-- hash: c770159c575ceb1fb6989ed0a07fe3b4b8c6bead95f64e6fe17a1420a10a7189++name:           hpc-threshold+version:        0.1.0.0+synopsis:       Small utility for validating whether HPC result is above defined thresholds+description:    Please see the README on Github at <https://github.com/eckyputrady/hpc-threshold#readme>+category:       Development+homepage:       https://github.com/eckyputrady/hpc-threshold#readme+bug-reports:    https://github.com/eckyputrady/hpc-threshold/issues+author:         Ecky Putrady+maintainer:     eckyputrady@gmail.com+copyright:      2018 Ecky Putrady+license:        BSD3+license-file:   LICENSE+build-type:     Simple+cabal-version:  >= 1.10++extra-source-files:+    ChangeLog.md+    README.md++source-repository head+  type: git+  location: https://github.com/eckyputrady/hpc-threshold++library+  hs-source-dirs:+      src+  default-extensions: OverloadedStrings QuasiQuotes+  build-depends:+      base >=4.7 && <5+    , bytestring+    , interpolate+    , pcre-heavy+  exposed-modules:+      Lib+  other-modules:+      Paths_hpc_threshold+  default-language: Haskell2010++executable hpc-threshold+  main-is: Main.hs+  hs-source-dirs:+      app+  default-extensions: OverloadedStrings QuasiQuotes+  ghc-options: -threaded -rtsopts -with-rtsopts=-N+  build-depends:+      base >=4.7 && <5+    , hpc-threshold+  other-modules:+      Paths_hpc_threshold+  default-language: Haskell2010++test-suite hpc-threshold-test+  type: exitcode-stdio-1.0+  main-is: Spec.hs+  hs-source-dirs:+      test+  default-extensions: OverloadedStrings QuasiQuotes+  ghc-options: -threaded -rtsopts -with-rtsopts=-N+  build-depends:+      base >=4.7 && <5+    , hpc-threshold+  other-modules:+      Paths_hpc_threshold+  default-language: Haskell2010

@@ -0,0 +1,69 @@+module Lib where++import Text.Regex.PCRE.Heavy+import Data.ByteString (ByteString)+import Data.String.Interpolate+import System.Exit++data Threshold = Threshold+  { thresholdName :: String+  , thresholdRegex :: ByteString+  , thresholdValue :: Double+  } deriving (Read, Show)++type Coverage = Double+type ThresholdEvaluationResult = (Threshold, Coverage, Bool)++extractCoverage :: String -> ByteString -> Double+extractCoverage src regexStr =+  case compileM regexStr [] of+    Left e ->+      error e+    Right regex ->+      case scan regex src of+        (_, coverage:_):_ ->+          read coverage+        _ ->+          0++evaluate :: String -> [Threshold] -> [ThresholdEvaluationResult]+evaluate src thresholds = do+  threshold <- thresholds+  let +    coverage =+      extractCoverage src (thresholdRegex threshold)+    isPass =+      coverage >= thresholdValue threshold+  return (threshold, coverage, isPass)++reportThreshold :: ThresholdEvaluationResult -> String+reportThreshold (Threshold tName _ tValue, coverage, isPass) =+  let+    remark = if isPass then "✓" else "·"+    sign = if isPass then "≥" else "<"+  in+    [i|#{remark} #{tName}: #{coverage}% (#{sign} #{tValue}%)|]++evaluateAndReport :: String -> [Threshold] -> (String, Bool)+evaluateAndReport src thresholds =+  let+    evalResults = evaluate src thresholds+    isAllThresholdPass = all (\(_, _, isPass) -> isPass) evalResults+    reportSummary =+      if isAllThresholdPass+        then "Code coverage threshold check: PASS"+        else "Code coverage threshold check: FAIL"+    report = unlines $ reportSummary : map reportThreshold evalResults+  in+    (report, isAllThresholdPass)++evaluateAndReport' :: IO ()+evaluateAndReport' = do+  thresholds <- readFile ".hpc-threshold" >>= return . read+  src <- getContents+  let (report, isPass) = evaluateAndReport src thresholds+  putStrLn report+  if isPass+    then exitWith ExitSuccess+    else exitWith $ ExitFailure 1+

@@ -0,0 +1,2 @@+main :: IO ()+main = putStrLn "Test suite not yet implemented"