hsbencher-1.3.9: hsbencher.cabal
name: hsbencher
version: 1.3.9
-- CHANGELOG:
-- 1.0 : Initial release, new flexible benchmark format.
-- 1.1 : Change interface to RunInPlace
-- 1.1.1 : add defaultMainModifyConfig
-- 1.2 : Significant interface changes.
-- 1.2.1 : Hack to shorten tuples, annoying URL length problem.
-- 1.3 : breaking change to Config/RunResult types
-- 1.3.1 : bugfix
-- 1.3.2 : change policy for timeouts and errors
-- 1.3.4 : Add ability to prune/select benchmarks with commmand line args
-- 1.3.6 : bugfix productivity for timeouts
-- 1.3.8 : Added --skipto and --runid
-- 1.3.8 : Remove hydra-print dep by default. Add 'hydra' flag.
synopsis: Flexible benchmark runner for Haskell and non-Haskell benchmarks.
description: Benchmark frameworks are usually very specific to the
host language/environment. Hence they are usually about as reusable
as compiler passes (that is, not).
.
Nevertheless, `hsbencher` is an attempt at a reusable benchmark
framework. It knows fairly little about what the benchmarks do, and
is mostly concerned with defining and iterating through
configuration spaces (e.g. varying the number of threads), and
managing the data that results.
.
Benchmark data is stored in simple text files, and optionally
uploaded to Google Fusion Tables.
-- TODO: Describe clusterbench functionality when it's ready.
.
`hsbencher` attempts to stradle the divide between language-specific
and language-agnostic by having an extensible set of `BuildMethod`s.
As shipped, `hsbencher` knows a little about cabal, ghc, and less
about Make, but it can be taught more.
.
The general philosophy is to have benchmarks follow a simple
protocol, for example printing out a line "SELFTIMED: 3.3s" if they
wish to report their own timing. The focus is on benchmarks that
run long enough to run in their own process. This is typical of
parallelism benchmarks and different than the fine grained
benchmarks that are well supported by "Criterion".
.
.
`hsbencher` is used by creating a script or executable that imports `HSBencher`
and provides a list of benchmarks, each of which is decorated with its
parameter space. Below is a minimal example that creates a two-configuration
parameter space:
.
@
import HSBencher
main = defaultMainWithBechmarks
. [ Benchmark \"bench1/bench1.cabal\" [\"1000\"] $
. Or [ Set NoMeaning (RuntimeParam \"+RTS -qa -RTS\")
. , Set NoMeaning (RuntimeEnv \"HELLO\" \"yes\") ] ]
@
.
The output would appear as in this gist:
<https://gist.github.com/rrnewton/5667800>
.
More examples can be found here:
<https://github.com/rrnewton/HSBencher/tree/master/example>
.
ChangesLog:
.
* (1.3.8) Added @--skipto@ and @--runid@ arguments
.
* (1.3.4) Added ability to prune benchmarks with patterns on command line.
license: BSD3
license-file: LICENSE
author: Ryan Newton
maintainer: rrnewton@gmail.com
copyright: (c) Ryan Newton 2013
category: Development
build-type: Simple
cabal-version: >=1.10
extra-source-files: README.md
example/make_and_ghc/runit.sh
example/make_and_ghc/benchmark.hs
example/make_and_ghc/bench1/Makefile
example/make_and_ghc/bench1/hello.c
example/make_and_ghc/bench2/Hello.hs
example/cabal/runit.sh
example/cabal/benchmark.hs
example/cabal/bench1/bench1.cabal
example/cabal/bench1/Hello.hs
Flag fusion
description:
Add support for Google Fusion Table upload of benchmark data.
default: True
Flag hydra
description:
Add support for (and dependency on) the hydra-print library.
default: False
Library
source-repository head
type: git
location: https://github.com/rrnewton/HSBenchScaling.git
exposed-modules: HSBencher
HSBencher.App
HSBencher.Types
HSBencher.Config
HSBencher.Logging
HSBencher.Methods
HSBencher.Utils
HSBencher.MeasureProcess
other-modules: Paths_hsbencher
build-depends:
-- base ==4.6.*, bytestring ==0.10.*, process ==1.1.*, directory ==1.2.*, filepath ==1.3.*, random ==1.0.*,
-- unix ==2.6.*, containers ==0.5.*, time ==1.4.*, mtl ==2.1.*, async >= 2.0,
base >= 4.5 && <= 4.7, bytestring, process, directory, filepath, random, unix, containers, time, mtl, async,
io-streams >= 1.1,
GenericPretty >= 1.2, http-conduit
if flag(hydra) {
build-depends: hydra-print >= 0.1.0.3
cpp-options: -DUSE_HYDRAPRINT
}
default-language: Haskell2010
if flag(fusion) {
build-depends: handa-gdata >= 0.6.2
exposed-modules: HSBencher.Fusion
cpp-options: -DFUSION_TABLES
}
-- [2013.05.28] This will come back later when the new ASCII benchmark file format is finished:
-----------------------------------------------------------------------------------------------
-- Executable hsbencher
-- main-is: Main.hs
-- -- other-modules:
-- build-depends:
-- -- <DUPLICATED from above>
-- base >= 4.5, bytestring, process, directory, filepath, random, unix, containers, time, mtl, async,
-- hydra-print >= 0.1.0.3, io-streams >= 1.0,
-- GenericPretty >= 1.2
-- -- </DUPLICATED>
-- ghc-options: -threaded
-- if flag(fusion) {
-- build-depends: handa-gdata >= 0.6.2
-- cpp-options: -DFUSION_TABLES
-- }
-- default-language: Haskell2010
Test-suite test1
main-is: example/cabal/benchmark.hs
type: exitcode-stdio-1.0
build-depends:
-- <DUPLICATED from above>
base >= 4.5 && <= 4.7, bytestring, process, directory, filepath, random, unix, containers, time, mtl, async,
io-streams >= 1.0,
GenericPretty >= 1.2, http-conduit, hsbencher
-- </DUPLICATED>
ghc-options: -threaded
default-language: Haskell2010
if flag(hydra) {
build-depends: hydra-print >= 0.1.0.3
}
if flag(fusion) {
build-depends: handa-gdata >= 0.6.2
cpp-options: -DFUSION_TABLES
}
Test-suite test2
main-is: example/make_and_ghc/benchmark.hs
type: exitcode-stdio-1.0
build-depends:
-- <DUPLICATED from above>
base >= 4.5 && <= 4.7, bytestring, process, directory, filepath, random, unix, containers, time, mtl, async,
io-streams >= 1.0,
GenericPretty >= 1.2, http-conduit, hsbencher
-- </DUPLICATED>
ghc-options: -threaded
default-language: Haskell2010
if flag(hydra) {
build-depends: hydra-print >= 0.1.0.3
}
if flag(fusion) {
build-depends: handa-gdata >= 0.6.2
cpp-options: -DFUSION_TABLES
}