hsbencher-1.2: hsbencher.cabal
name: hsbencher
version: 1.2
-- CHANGELOG:
-- 1.0 : Initial release, new flexible benchmark format.
-- 1.1 : Change interface to RunInPlace
-- 1.1.1 : add defaultMainModifyConfig
-- 1.2 : Significant interface changes.
synopsis: Flexible benchmark runner for Haskell and non-Haskell benchmarks.
description: Benchmark frameworks are usually very specific to the
host language/environment. Hence they are usually about as reusable
as compiler passes (that is, not).
.
Nevertheless, `hsbencher` is an attempt at a reusable benchmark
framework. It knows fairly little about what the benchmarks do, and
is mostly concerned with defining and iterating through
configuration spaces (e.g. varying the number of threads), and
managing the data that results.
.
Benchmark data is stored in simple text files, and optionally
uploaded to Google Fusion Tables.
-- TODO: Describe clusterbench functionality when it's ready.
.
`hsbencher` attempts to stradle the divide between language-specific
and language-agnostic by having an extensible set of `BuildMethod`s.
As shipped, `hsbencher` knows a little about cabal, ghc, and less
about Make, but it can be taught more.
.
The general philosophy is to have benchmarks follow a simple
protocol, for example printing out a line "SELFTIMED: 3.3s" if they
wish to report their own timing. The focus is on benchmarks that
run long enough to run in their own process. This is typical of
parallelism benchmarks and different than the fine grained
benchmarks that are well supported by "Criterion".
.
.
`hsbencher` is used by creating a script or executable that imports `HSBencher`
and provides a list of benchmarks, each of which is decorated with its
parameter space. Below is a minimal example that creates a two-configuration
parameter space:
.
@
import HSBencher
main = defaultMainWithBechmarks
. [ Benchmark "bench1/bench1.cabal" ["1000"] $
. Or [ Set NoMeaning (RuntimeParam "+RTS -qa -RTS")
. , Set NoMeaning (RuntimeEnv "HELLO" "yes") ] ]
@
.
The output would appear as in this gist:
<https://gist.github.com/rrnewton/5667800>
.
More examples can be found here:
<https://github.com/rrnewton/HSBencher/tree/master/example>
license: BSD3
license-file: LICENSE
author: Ryan Newton
maintainer: rrnewton@gmail.com
copyright: (c) Ryan Newton 2013
category: Development
build-type: Simple
cabal-version: >=1.10
extra-source-files: README.md
example/make_and_ghc/runit.sh
example/make_and_ghc/benchmark.hs
example/make_and_ghc/bench1/Makefile
example/make_and_ghc/bench1/hello.c
example/make_and_ghc/bench2/Hello.hs
example/cabal/runit.sh
example/cabal/benchmark.hs
example/cabal/bench1/bench1.cabal
example/cabal/bench1/Hello.hs
Flag fusion
description:
Add support for Google Fusion Table upload of benchmark data.
default: True
Library
source-repository head
type: git
location: https://github.com/rrnewton/HSBenchScaling.git
exposed-modules: HSBencher
HSBencher.App
HSBencher.Types
HSBencher.Config
HSBencher.Logging
HSBencher.Methods
HSBencher.Utils
HSBencher.MeasureProcess
other-modules: Paths_hsbencher
build-depends:
-- base ==4.6.*, bytestring ==0.10.*, process ==1.1.*, directory ==1.2.*, filepath ==1.3.*, random ==1.0.*,
-- unix ==2.6.*, containers ==0.5.*, time ==1.4.*, mtl ==2.1.*, async >= 2.0,
base >= 4.5 && <= 4.7, bytestring, process, directory, filepath, random, unix, containers, time, mtl, async,
hydra-print >= 0.1.0.3, io-streams >= 1.0,
GenericPretty >= 1.2, http-conduit
default-language: Haskell2010
if flag(fusion) {
build-depends: handa-gdata >= 0.6.2
exposed-modules: HSBencher.Fusion
cpp-options: -DFUSION_TABLES
}
-- [2013.05.28] This will come back later when the new ASCII benchmark file format is finished:
-----------------------------------------------------------------------------------------------
-- Executable hsbencher
-- main-is: Main.hs
-- -- other-modules:
-- build-depends:
-- -- <DUPLICATED from above>
-- base >= 4.5, bytestring, process, directory, filepath, random, unix, containers, time, mtl, async,
-- hydra-print >= 0.1.0.3, io-streams >= 1.0,
-- GenericPretty >= 1.2
-- -- </DUPLICATED>
-- ghc-options: -threaded
-- if flag(fusion) {
-- build-depends: handa-gdata >= 0.6.2
-- cpp-options: -DFUSION_TABLES
-- }
-- default-language: Haskell2010
Test-suite test1
main-is: example/cabal/benchmark.hs
type: exitcode-stdio-1.0
build-depends:
-- <DUPLICATED from above>
base >= 4.5 && <= 4.7, bytestring, process, directory, filepath, random, unix, containers, time, mtl, async,
hydra-print >= 0.1.0.3, io-streams >= 1.0,
GenericPretty >= 1.2, http-conduit, hsbencher
-- </DUPLICATED>
ghc-options: -threaded
default-language: Haskell2010
if flag(fusion) {
build-depends: handa-gdata >= 0.6.2
cpp-options: -DFUSION_TABLES
}
Test-suite test2
main-is: example/make_and_ghc/benchmark.hs
type: exitcode-stdio-1.0
build-depends:
-- <DUPLICATED from above>
base >= 4.5 && <= 4.7, bytestring, process, directory, filepath, random, unix, containers, time, mtl, async,
hydra-print >= 0.1.0.3, io-streams >= 1.0,
GenericPretty >= 1.2, http-conduit, hsbencher
-- </DUPLICATED>
ghc-options: -threaded
default-language: Haskell2010
if flag(fusion) {
build-depends: handa-gdata >= 0.6.2
cpp-options: -DFUSION_TABLES
}