* subprocess communication is getting stuck
. But why? I think this should be the same code as karya's version, which
doesn't do that.
. There's definitely something fishy going on in both cases though, since
karya's version sometimes fails on travis.
. dist/build/test/test --clear-dirs --output=dist/test-output .
How to run GenerateRunTests?
- use physical CPUs instead of getNumCapabilities
. The cpuinfo package is meant to do this but it's linux only:
https://github.com/TravisWhitaker/cpuinfo/issues/1
* There should be a standard set of "run from cabal" flags:
--clear-dir --output=dist/test-output --check-output .
/ Makefile to update when out of date, cabal Setup hooks to run make.
/ Like Makefile, but a shell script that does the same thing.
/ Use Setup hooks directly. Does cabal know how to only rebuild when
necessary?
* HSpec uses a -pgmF program to generate the module. But I think it runs
every single time you build tests. Of course, maybe that's actually pretty
fast.
. pgmF args: OrigSourceName.hs input.hs output.hs
* remove the ioHuman and interactive stuff
/ argv0 is wrong, so parallelization probably doesn't work
. Use Environment.getExecutablePath
* fix tmpBaseDir
. Surely cabal has some way to pass in a tmp dir?
. Haha, it's cabal of course not. Use CPP then.
. Well, it would have to come from the testee config, which means putting it
in Testing.Config. Too much bother, just hardcode to dist/build/tmp-test
Interpret the output
. I need the equivalent of test/run_tests.
. Can I embed that logic in RunTest itself? The problem is I want to write
to stdout, and then check that stdout. It's hard to do that when cabal is
redirecting stdout.
. Does cabal have any more flexible way to run tests?
Looks like no, just 'detailed', which is even less flexible, since it
requires the tests to be some cabal data structure.
. I would like a runner that runs the test binary, then runs something else
with access to the log.
. Can I redirect stdout to a file temporarily? I could do
withStdout (open "output") $ runTests ...
check "output"
. I can have tests write to a file via 'success' and 'failure'. The problem
is that then debugging isn't interleaved. But I could write to both the
output file and stdout. Or write everything to sdout, only errors to
stderr.
. Or use the parallelization feature, run as a subprocess.
. How this works is you call with --outputs. Then RunTests can have a flag
to check the outputs afterwards.