diff --git a/LICENSE b/LICENSE new file mode 100644 --- /dev/null +++ b/LICENSE @@ -0,0 +1,30 @@ +Copyright Gabriel Gonzalez (c) 2016 + +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above + copyright notice, this list of conditions and the following + disclaimer in the documentation and/or other materials provided + with the distribution. + + * Neither the name of nor the names of other + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/README.md b/README.md new file mode 100644 --- /dev/null +++ b/README.md @@ -0,0 +1,122 @@ +# Bench v1.0.0 + +Think of this as a more powerful alternative to the `time` command. Use this +command-line tool to benchmark a command using Haskell's `criterion` library. + +Key features: + +* Repeated runs +* Detailed statistical output +* Gorgeous HTML output (via the `--output` flag) + ([Example](http://www.serpentine.com/criterion/fibber.html)) +* Also supports CSV or templated output + +## Quick Start + +First, download the +[Haskell toolchain](https://www.haskell.org/downloads#minimal), which provides +the `stack` build tool. + +You will also need to download [git](https://git-scm.com/downloads) if you +haven't done so already. + +Then run the following commands from a terminal to build and install the `bench` +tool: + +```bash +$ git clone https://github.com/Gabriel439/bench.git +$ cd bench +$ stack install +``` + +`stack install` will install `bench` to `~/.local/bin` or something similar. +Make sure that the installation directory is on your executable search path +before running the following command. `stack` will remind you to do this if you +forget. + +```bash +$ bench 'sleep 1' # Don't forget to quote the command line +benchmarking sleep 1 +time 1.003 s (1.002 s .. 1.003 s) + 1.000 R² (1.000 R² .. 1.000 R²) +mean 1.003 s (1.003 s .. 1.003 s) +std dev 92.92 μs (0.0 s .. 101.8 μs) +variance introduced by outliers: 19% (moderately inflated) +``` + +All output from the command being benchmarked is discarded + +## Usage + +``` +$ bench --help +Command-line tool to benchmark other programs + +Usage: bench COMMAND ([-I|--ci CI] [-G|--no-gc] [-L|--time-limit SECS] + [--resamples COUNT] [--regress RESP:PRED..] [--raw FILE] + [-o|--output FILE] [--csv FILE] [--junit FILE] + [-v|--verbosity LEVEL] [-t|--template FILE] [-m|--match MATCH] + [NAME...] | [-n|--iters ITERS] [-m|--match MATCH] [NAME...] | + [-l|--list] | [--version]) + +Available options: + -h,--help Show this help text + COMMAND The command line to benchmark + -I,--ci CI Confidence interval + -G,--no-gc Do not collect garbage between iterations + -L,--time-limit SECS Time limit to run a benchmark + --resamples COUNT Number of bootstrap resamples to perform + --regress RESP:PRED.. Regressions to perform + --raw FILE File to write raw data to + -o,--output FILE File to write report to + --csv FILE File to write CSV summary to + --junit FILE File to write JUnit summary to + -v,--verbosity LEVEL Verbosity level + -t,--template FILE Template to use for report + -m,--match MATCH How to match benchmark names ("prefix" or "glob") + -n,--iters ITERS Run benchmarks, don't analyse + -m,--match MATCH How to match benchmark names ("prefix" or "glob") + -l,--list List benchmarks + --version Show version info +``` + +## Development Status + +[![Build Status](https://travis-ci.org/Gabriel439/bench.png)](https://travis-ci.org/Gabriel439/bench) + +This is a pretty simple utility which just wraps `criterion` in a command-line +tool, so I don't expect this project to change much. However, only time will +tell. + +## License (BSD 3-clause) + +Copyright Gabriel Gonzalez (c) 2016 + +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +* Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + +* Redistributions in binary form must reproduce the above + copyright notice, this list of conditions and the following + disclaimer in the documentation and/or other materials provided + with the distribution. + +* Neither the name of nor the names of other + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/Setup.hs b/Setup.hs new file mode 100644 --- /dev/null +++ b/Setup.hs @@ -0,0 +1,2 @@ +import Distribution.Simple +main = defaultMain diff --git a/bench.cabal b/bench.cabal new file mode 100644 --- /dev/null +++ b/bench.cabal @@ -0,0 +1,32 @@ +name: bench +version: 1.0.0 +synopsis: Command-line benchmark tool +description: Think of this as a more powerful alternative to the @time@ + command. Use this command-line tool to benchmark a command + using Haskell's @criterion@ library. +homepage: http://github.com/Gabriel439/bench +bug-reports: https://github.com/Gabriel439/bench/issues +license: BSD3 +license-file: LICENSE +author: Gabriel Gonzalez +maintainer: Gabriel439@gmail.com +copyright: 2016 Gabriel Gonzalez +category: System +build-type: Simple +cabal-version: >=1.10 +extra-source-files: + README.md +source-repository head + type: git + location: https://github.com/Gabriel439/bench + +executable bench + hs-source-dirs: src + main-is: Main.hs + default-language: Haskell2010 + build-depends: base >= 4.5 && < 5 + , criterion >= 1.0.0.1 && < 1.2 + , text < 1.3 + , silently >= 1.1.1 && < 1.3 + , turtle >= 1.2.5 && < 1.3 + ghc-options: -Wall -O2 -threaded diff --git a/src/Main.hs b/src/Main.hs new file mode 100644 --- /dev/null +++ b/src/Main.hs @@ -0,0 +1,34 @@ +{-# LANGUAGE OverloadedStrings #-} + +module Main where + +import Control.Applicative +import Data.Text (Text) +import Turtle (Parser) + +import qualified Criterion +import qualified Criterion.Main as Criterion +import qualified Criterion.Main.Options as Criterion +import qualified Data.Text as Text +import qualified System.IO as IO +import qualified System.IO.Silently as Silently +import qualified Turtle + +data Options = Options + { command :: Text + , mode :: Criterion.Mode + } + +parser :: Parser Options +parser = + Options + <$> Turtle.argText "command" "The command line to benchmark" + <*> Criterion.parseWith Criterion.defaultConfig + +main :: IO () +main = do + o <- Turtle.options "Command-line tool to benchmark other programs" parser + let io = Turtle.shells (command o) empty + let benchmark = Criterion.nfIO (Silently.hSilence [IO.stdout, IO.stderr] io) + let name = Text.unpack (command o) + Criterion.runMode (mode o) [ Criterion.bench name benchmark ]