diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -2,7 +2,7 @@ ## Bug Reports -Before reporting a bug, please ensure that you are using the latest release (currently stack-1.0.4). See the [upgrade instructions](http://docs.haskellstack.org/en/stable/install_and_upgrade/#upgrade) to upgrade. +Before reporting a bug, please ensure that you are using the latest release (currently stack-1.1.0). See the [upgrade instructions](http://docs.haskellstack.org/en/stable/install_and_upgrade/#upgrade) to upgrade. When reporting a bug, please write in the following format: diff --git a/ChangeLog.md b/ChangeLog.md --- a/ChangeLog.md +++ b/ChangeLog.md @@ -1,5 +1,140 @@ # Changelog +## v1.1.0 + +Release notes: + +* Added Ubuntu 16.04 LTS (xenial) Apt repo. +* No longer uploading new versions to Fedora 21 repo. + +Behavior changes: + +* Snapshot packages are no longer built with executable profiling. See + [#1179](https://github.com/commercialhaskell/stack/issues/1179). +* `stack init` now ignores symlinks when searching for cabal files. It also now + ignores any directory that begins with `.` (as well as `dist` dirs) - before + it would only ignore `.git`, `.stack-work`, and `dist`. +* The stack executable is no longer built with `-rtsopts`. Before, when + `-rtsopts` was enabled, stack would process `+RTS` options even when intended + for some other program, such as when used with `stack exec -- prog +RTS`. + See [#2022](https://github.com/commercialhaskell/stack/issues/2022). +* The `stack path --ghc-paths` option is deprecated and renamed to `--programs`. + `--compiler` is added, which points directly at the compiler used in + the current project. `--compiler-bin` points to the compiler's bin dir. +* For consistency with the `$STACK_ROOT` environment variable, the + `stack path --global-stack-root` flag and the `global-stack-root` field + in the output of `stack path` are being deprecated and replaced with the + `stack-root` flag and output field. + Additionally, the stack root can now be specified via the + `--stack-root` command-line flag. See + [#1148](https://github.com/commercialhaskell/stack/issues/1148). +* `stack sig` GPG-related sub-commands were removed (folded into `upload` and + `sdist`) +* GPG signing of packages while uploading to Hackage is now the default. Use + `upload --no-signature` if you would rather not contribute your package + signature. If you don't yet have a GPG keyset, read this + [blog post on GPG keys](https://fpcomplete.com/blog/2016/04/stack-security-gnupg-keys) + We can add a stack.yaml config setting to disable signing if some people + desire it. We hope that people will sign. Later we will be adding GPG + signature verification options. +* `stack build pkg-1.2.3` will now build even if the snapshot has a different + package version - it is treated as an extra-dep. `stack build local-pkg-1.2.3` + is an error even if the version number matches the local package + [#2028](https://github.com/commercialhaskell/stack/issues/2028). +* Having a `nix:` section no longer implies enabling nix build. This allows the + user to globally configure whether nix is used (unless the project overrides + the default explicitly). See + [#1924](https://github.com/commercialhaskell/stack/issues/1924). +* Remove deprecated valid-wanted field. +* Docker: mount home directory in container [#1949](https://github.com/commercialhaskell/stack/issues/1949). +* Deprecate `--local-bin-path` instead `--local-bin`. +* `stack image`: allow absolute source paths for `add`. + +Other enhancements: + +* `stack haddock --open [PACKAGE]` opens the local haddocks in the browser. +* Fix too much rebuilding when enabling/disabling profiling flags. +* `stack build pkg-1.0` will now build `pkg-1.0` even if the snapshot specifies + a different version (it introduces a temporary extra-dep) +* Experimental support for `--split-objs` added + [#1284](https://github.com/commercialhaskell/stack/issues/1284). +* `git` packages with submodules are supported by passing the `--recursive` + flag to `git clone`. +* When using [hpack](https://github.com/sol/hpack), only regenerate cabal files + when hpack files change. +* hpack files can now be used in templates +* `stack ghci` now runs ghci as a separate process + [#1306](https://github.com/commercialhaskell/stack/issues/1306) +* Retry when downloading snapshots and package indices +* Many build options are configurable now in `stack.yaml`: +``` + build: + library-profiling: true + executable-profiling: true + haddock: true + haddock-deps: true + copy-bins: true + prefetch: true + force-dirty: true + keep-going: true + test: true + test-arguments: + rerun-tests: true + additional-args: ['-fprof'] + coverage: true + no-run-tests: true + bench: true + benchmark-opts: + benchmark-arguments: -O2 + no-run-benchmarks: true + reconfigure: true + cabal-verbose: true +``` +* A number of URLs are now configurable, useful for firewalls. See + [#1794](https://github.com/commercialhaskell/stack/issues/1884). +* Suggest causes when executables are missing. +* Allow `--omit-packages` even without `--solver`. +* Improve the generated stack.yaml. +* Improve ghci results after :load Main module collision with main file path. +* Only load the hackage index if necessary + [#1883](https://github.com/commercialhaskell/stack/issues/1883), [#1892](https://github.com/commercialhaskell/stack/issues/1892). +* init: allow local packages to be deps of deps + [#1965](https://github.com/commercialhaskell/stack/issues/1965). +* Always use full fingerprints from GPG + [#1952](https://github.com/commercialhaskell/stack/issues/1952). +* Default to using `gpg2` and fall back to `gpg` + [#1976](https://github.com/commercialhaskell/stack/issues/1976). +* Add a flag for --verbosity silent. +* Add `haddock --open` flag [#1396](https://github.com/commercialhaskell/stack/issues/1396). + +Bug fixes: + +* Package tarballs would fail to unpack. + [#1884](https://github.com/commercialhaskell/stack/issues/1884). +* Fixed errant warnings about missing modules, after deleted and removed from + cabal file [#921](https://github.com/commercialhaskell/stack/issues/921) + [#1805](https://github.com/commercialhaskell/stack/issues/1805). +* Now considers a package to dirty when the hpack file is changed + [#1819](https://github.com/commercialhaskell/stack/issues/1819). +* Nix: cancelling a stack build now exits properly rather than dropping into a + nix-shell [#1778](https://github.com/commercialhaskell/stack/issues/1778). +* `allow-newer: true` now causes `--exact-configuration` to be passed to Cabal. + See [#1579](https://github.com/commercialhaskell/stack/issues/1579). +* `stack solver` no longer fails with `InvalidRelFile` for relative package + paths including `..`. See + [#1954](https://github.com/commercialhaskell/stack/issues/1954). +* Ignore emacs lock files when finding .cabal + [#1897](https://github.com/commercialhaskell/stack/issues/1897). +* Use lenient UTF-8 decode for build output + [#1945](https://github.com/commercialhaskell/stack/issues/1945). +* Clear index cache whenever index updated + [#1962](https://github.com/commercialhaskell/stack/issues/1962). +* Fix: Building a container image drops a .stack-work dir in the current working + (sub)directory + [#1975](https://github.com/commercialhaskell/stack/issues/1975). +* Fix: Rebuilding when disabling profiling + [#2023](https://github.com/commercialhaskell/stack/issues/2023). + ## 1.0.4.3 Bug fixes: @@ -42,7 +177,7 @@ work [#1358](https://github.com/commercialhaskell/stack/issues/1358). * Docker: strip suffix from docker --version. [#1653](https://github.com/commercialhaskell/stack/issues/1653) -* Docker: pass USER and PWD environment bariables into container. +* Docker: pass USER and PWD environment variables into container. * On each run, stack will test the stack root directory (~/.stack), and the project and package work directories (.stack-work) for whether they are owned by the current user and abort if they are not. This precaution can @@ -347,6 +482,8 @@ * `stack new` disallows package names with "words" consisting solely of numbers [#1336](https://github.com/commercialhaskell/stack/issues/1336) * `stack build --fast` turns off optimizations +* Show progress while downloading package index + [#1223](https://github.com/commercialhaskell/stack/issues/1223). Bug fixes: diff --git a/README.md b/README.md --- a/README.md +++ b/README.md @@ -4,8 +4,7 @@ [![Windows build status](https://ci.appveyor.com/api/projects/status/c1c7uvmw6x1dupcl?svg=true)](https://ci.appveyor.com/project/snoyberg/stack) [![Release](https://img.shields.io/github/release/commercialhaskell/stack.svg)](https://github.com/commercialhaskell/stack/releases) -`stack` is a cross-platform program for developing Haskell projects. It is aimed -at Haskellers both new and experienced. +Stack is a cross-platform program for developing Haskell projects. It is intended for Haskellers both new and experienced. See [haskellstack.org](http://haskellstack.org) or the `doc` directory for more information. diff --git a/doc/CONTRIBUTING.md b/doc/CONTRIBUTING.md new file mode 100644 --- /dev/null +++ b/doc/CONTRIBUTING.md @@ -0,0 +1,79 @@ +# Contributors Guide + +## Bug Reports + +Before reporting a bug, please ensure that you are using the latest release (currently stack-1.1.0). See the [upgrade instructions](http://docs.haskellstack.org/en/stable/install_and_upgrade/#upgrade) to upgrade. + +When reporting a bug, please write in the following format: + +> [Any general summary/comments if desired] + +> **Steps to reproduce:** + +> 1. _Remove directory *blah*._ +> 2. _Run command `stack blah`._ +> 3. _Edit file blah._ +> 3. _Run command `stack blah`._ + +> **Expected:** + +> _What I expected to see and happen._ + +> **Actual:** + +> _What actually happened._ +> +> Here is the `stack --version` output: +> +> ``` +> $ stack --version +> Version 0.0.2, Git revision 6a86ee32e5b869a877151f74064572225e1a0398 +> ``` +> Here is the command I ran **with `--verbose`**: +> +> ``` +> $ stack --verbose +> +> ``` + +With `--verbose` mode we can see what the tool is doing and when. Without this output it is much more difficult to surmise what's going on with your issue. If the above output is larger than a page, paste it in a private [Gist](https://gist.github.com/) instead. + +Include any `.yaml` configuration if relevant. + +The more detailed your report, the faster it can be resolved and will ensure it is resolved in the right way. Once your bug has been resolved, the responsible will tag the issue as _Needs confirmation_ and assign the issue back to you. Once you have tested and confirmed that the issue is resolved, close the issue. If you are not a member of the project, you will be asked for confirmation and we will close it. + + +## Documentation + +If you would like to help with documentation, please note that for most cases the Wiki has been deprecated in favor of markdown files placed in a new `/doc` subdirectory of the repository itself. Please submit a [pull request](https://help.github.com/articles/using-pull-requests/) with your changes/additions. + +The documentation is rendered on [haskellstack.org](http://haskellstack.org) by +readthedocs.org using Sphinx and CommonMark. Since links and formatting vary +from GFM, please check the documentation there before submitting a PR to fix +those. In particular, links to other documentation files intentionally have +`.html` extensions instead of `.md`, unfortunately (see +[#1506](https://github.com/commercialhaskell/stack/issues/1506) for details). + +If your changes move or rename files, or subsume Wiki content, please continue to leave a file/page in the old location temporarily, in addition to the new location. This will allow users time to update any shared links to the old location. Please also update any links in other files, or on the Wiki, to point to the new file location. + + +## Code + +If you would like to contribute code to fix a bug, add a new feature, or +otherwise improve `stack`, pull requests are most welcome. It's a good idea to +[submit an issue](https://github.com/commercialhaskell/stack/issues/new) to +discuss the change before plowing into writing code. + +If you'd like to help out but aren't sure what to work on, look for issues with +the +[awaiting pr](https://github.com/commercialhaskell/stack/issues?q=is%3Aopen+is%3Aissue+label%3A%22awaiting+pr%22) +label. Issues that are suitable for newcomers to the codebase have the +[newcomer](https://github.com/commercialhaskell/stack/issues?q=is%3Aopen+is%3Aissue+label%3A%22awaiting+pr%22+label%3Anewcomer) +label. Best to post a comment to the issue before you start work, in case anyone +has already started. + +Please include a +[ChangeLog](https://github.com/commercialhaskell/stack/blob/master/ChangeLog.md) +entry and +[documentation](https://github.com/commercialhaskell/stack/tree/master/doc/) +updates with your pull request. diff --git a/doc/ChangeLog.md b/doc/ChangeLog.md new file mode 100644 --- /dev/null +++ b/doc/ChangeLog.md @@ -0,0 +1,769 @@ +# Changelog + +## v1.1.0 + +Release notes: + +* Added Ubuntu 16.04 LTS (xenial) Apt repo. +* No longer uploading new versions to Fedora 21 repo. + +Behavior changes: + +* Snapshot packages are no longer built with executable profiling. See + [#1179](https://github.com/commercialhaskell/stack/issues/1179). +* `stack init` now ignores symlinks when searching for cabal files. It also now + ignores any directory that begins with `.` (as well as `dist` dirs) - before + it would only ignore `.git`, `.stack-work`, and `dist`. +* The stack executable is no longer built with `-rtsopts`. Before, when + `-rtsopts` was enabled, stack would process `+RTS` options even when intended + for some other program, such as when used with `stack exec -- prog +RTS`. + See [#2022](https://github.com/commercialhaskell/stack/issues/2022). +* The `stack path --ghc-paths` option is deprecated and renamed to `--programs`. + `--compiler` is added, which points directly at the compiler used in + the current project. `--compiler-bin` points to the compiler's bin dir. +* For consistency with the `$STACK_ROOT` environment variable, the + `stack path --global-stack-root` flag and the `global-stack-root` field + in the output of `stack path` are being deprecated and replaced with the + `stack-root` flag and output field. + Additionally, the stack root can now be specified via the + `--stack-root` command-line flag. See + [#1148](https://github.com/commercialhaskell/stack/issues/1148). +* `stack sig` GPG-related sub-commands were removed (folded into `upload` and + `sdist`) +* GPG signing of packages while uploading to Hackage is now the default. Use + `upload --no-signature` if you would rather not contribute your package + signature. If you don't yet have a GPG keyset, read this + [blog post on GPG keys](https://fpcomplete.com/blog/2016/04/stack-security-gnupg-keys) + We can add a stack.yaml config setting to disable signing if some people + desire it. We hope that people will sign. Later we will be adding GPG + signature verification options. +* `stack build pkg-1.2.3` will now build even if the snapshot has a different + package version - it is treated as an extra-dep. `stack build local-pkg-1.2.3` + is an error even if the version number matches the local package + [#2028](https://github.com/commercialhaskell/stack/issues/2028). +* Having a `nix:` section no longer implies enabling nix build. This allows the + user to globally configure whether nix is used (unless the project overrides + the default explicitly). See + [#1924](https://github.com/commercialhaskell/stack/issues/1924). +* Remove deprecated valid-wanted field. +* Docker: mount home directory in container [#1949](https://github.com/commercialhaskell/stack/issues/1949). +* Deprecate `--local-bin-path` instead `--local-bin`. +* `stack image`: allow absolute source paths for `add`. + +Other enhancements: + +* `stack haddock --open [PACKAGE]` opens the local haddocks in the browser. +* Fix too much rebuilding when enabling/disabling profiling flags. +* `stack build pkg-1.0` will now build `pkg-1.0` even if the snapshot specifies + a different version (it introduces a temporary extra-dep) +* Experimental support for `--split-objs` added + [#1284](https://github.com/commercialhaskell/stack/issues/1284). +* `git` packages with submodules are supported by passing the `--recursive` + flag to `git clone`. +* When using [hpack](https://github.com/sol/hpack), only regenerate cabal files + when hpack files change. +* hpack files can now be used in templates +* `stack ghci` now runs ghci as a separate process + [#1306](https://github.com/commercialhaskell/stack/issues/1306) +* Retry when downloading snapshots and package indices +* Many build options are configurable now in `stack.yaml`: +``` + build: + library-profiling: true + executable-profiling: true + haddock: true + haddock-deps: true + copy-bins: true + prefetch: true + force-dirty: true + keep-going: true + test: true + test-arguments: + rerun-tests: true + additional-args: ['-fprof'] + coverage: true + no-run-tests: true + bench: true + benchmark-opts: + benchmark-arguments: -O2 + no-run-benchmarks: true + reconfigure: true + cabal-verbose: true +``` +* A number of URLs are now configurable, useful for firewalls. See + [#1794](https://github.com/commercialhaskell/stack/issues/1884). +* Suggest causes when executables are missing. +* Allow `--omit-packages` even without `--solver`. +* Improve the generated stack.yaml. +* Improve ghci results after :load Main module collision with main file path. +* Only load the hackage index if necessary + [#1883](https://github.com/commercialhaskell/stack/issues/1883), [#1892](https://github.com/commercialhaskell/stack/issues/1892). +* init: allow local packages to be deps of deps + [#1965](https://github.com/commercialhaskell/stack/issues/1965). +* Always use full fingerprints from GPG + [#1952](https://github.com/commercialhaskell/stack/issues/1952). +* Default to using `gpg2` and fall back to `gpg` + [#1976](https://github.com/commercialhaskell/stack/issues/1976). +* Add a flag for --verbosity silent. +* Add `haddock --open` flag [#1396](https://github.com/commercialhaskell/stack/issues/1396). + +Bug fixes: + +* Package tarballs would fail to unpack. + [#1884](https://github.com/commercialhaskell/stack/issues/1884). +* Fixed errant warnings about missing modules, after deleted and removed from + cabal file [#921](https://github.com/commercialhaskell/stack/issues/921) + [#1805](https://github.com/commercialhaskell/stack/issues/1805). +* Now considers a package to dirty when the hpack file is changed + [#1819](https://github.com/commercialhaskell/stack/issues/1819). +* Nix: cancelling a stack build now exits properly rather than dropping into a + nix-shell [#1778](https://github.com/commercialhaskell/stack/issues/1778). +* `allow-newer: true` now causes `--exact-configuration` to be passed to Cabal. + See [#1579](https://github.com/commercialhaskell/stack/issues/1579). +* `stack solver` no longer fails with `InvalidRelFile` for relative package + paths including `..`. See + [#1954](https://github.com/commercialhaskell/stack/issues/1954). +* Ignore emacs lock files when finding .cabal + [#1897](https://github.com/commercialhaskell/stack/issues/1897). +* Use lenient UTF-8 decode for build output + [#1945](https://github.com/commercialhaskell/stack/issues/1945). +* Clear index cache whenever index updated + [#1962](https://github.com/commercialhaskell/stack/issues/1962). +* Fix: Building a container image drops a .stack-work dir in the current working + (sub)directory + [#1975](https://github.com/commercialhaskell/stack/issues/1975). +* Fix: Rebuilding when disabling profiling + [#2023](https://github.com/commercialhaskell/stack/issues/2023). + +## 1.0.4.3 + +Bug fixes: + +* Don't delete contents of ~/.ssh when using `stack clean --full` with Docker + enabled [#2000](https://github.com/commercialhaskell/stack/issues/2000) + +## 1.0.4.2 + +Build with path-io-1.0.0. There are no changes in behaviour from 1.0.4, +so no binaries are released for this version. + +## 1.0.4.1 + +Fixes build with aeson-0.11.0.0. There are no changes in behaviour from 1.0.4, +so no binaries are released for this version. + +## 1.0.4 + +Major changes: + +* Some notable changes in `stack init`: + * Overall it should now be able to initialize almost all existing cabal + packages out of the box as long as the package itself is consistently + defined. + * Choose the best possible snapshot and add extra dependencies on top + of a snapshot resolver rather than a compiler resolver - + [#1583](https://github.com/commercialhaskell/stack/pull/1583) + * Automatically omit a package (`--omit-packages`) when it is compiler + incompatible or when there are packages with conflicting dependency + requirements - [#1674](https://github.com/commercialhaskell/stack/pull/1674). + * Some more changes for a better user experience. Please refer to + the doc guide for details. +* Add support for hpack, alternative package description format + [#1679](https://github.com/commercialhaskell/stack/issues/1679) + +Other enhancements: + +* Docker: pass ~/.ssh and SSH auth socket into container, so that git repos + work [#1358](https://github.com/commercialhaskell/stack/issues/1358). +* Docker: strip suffix from docker --version. + [#1653](https://github.com/commercialhaskell/stack/issues/1653) +* Docker: pass USER and PWD environment variables into container. +* On each run, stack will test the stack root directory (~/.stack), and the + project and package work directories (.stack-work) for whether they are + owned by the current user and abort if they are not. This precaution can + be disabled with the `--allow-different-user` flag or `allow-different-user` + option in the global config (~/.stack/config.yaml). + [#471](https://github.com/commercialhaskell/stack/issues/471) +* Added `stack clean --full` option for full working dir cleanup. +* YAML config: support Zip archives. +* Redownload build plan if parsing fails + [#1702](https://github.com/commercialhaskell/stack/issues/1702). +* Give mustache templates access to a 'year' tag + [#1716](https://github.com/commercialhaskell/stack/pull/1716). +* Have "stack ghci" warn about module name aliasing. +* Add "stack ghci --load-local-deps". +* Build Setup.hs with -rtsopts + [#1687](https://github.com/commercialhaskell/stack/issues/1687). +* `stack init` accepts a list of directories. +* Add flag infos to DependencyPlanFailures (for better error output in case of + flags) [#713](https://github.com/commercialhaskell/stack/issues/713) +* `stack new --bare` complains for overwrites, and add `--force` option + [#1597](https://github.com/commercialhaskell/stack/issues/1597). + +Bug fixes: + +* Previously, `stack ghci` would fail with `cannot satisfy -package-id` when the + implicit build step changes the package key of some dependency. +* Fix: Building with ghcjs: "ghc-pkg: Prelude.chr: bad argument: 2980338" + [#1665](https://github.com/commercialhaskell/stack/issues/1665). +* Fix running test / bench with `--profile` / `--trace`. +* Fix: build progress counter is no longer visible + [#1685](https://github.com/commercialhaskell/stack/issues/1685). +* Use "-RTS" w/ profiling to allow extra args + [#1772](https://github.com/commercialhaskell/stack/issues/1772). +* Fix withUnpackedTarball7z to find name of srcDir after unpacking + (fixes `stack setup` fails for ghcjs project on windows) + [#1774](https://github.com/commercialhaskell/stack/issues/1774). +* Add space before auto-generated bench opts (makes profiling options work + uniformly for applications and benchmark suites) + [#1771](https://github.com/commercialhaskell/stack/issues/1771). +* Don't try to find plugin if it resembles flag. +* Setup.hs changes cause package dirtiness + [#1711](https://github.com/commercialhaskell/stack/issues/1711). +* Send "stack templates" output to stdout + [#1792](https://github.com/commercialhaskell/stack/issues/1792). + +## 1.0.2 + +Release notes: + +- Arch Linux: Stack has been adopted into the + [official community repository](https://www.archlinux.org/packages/community/x86_64/stack/), + so we will no longer be updating the AUR with new versions. See the + [install/upgrade guide](http://docs.haskellstack.org/en/stable/install_and_upgrade/#arch-linux) + for current download instructions. + +Major changes: + +- `stack init` and `solver` overhaul + [#1583](https://github.com/commercialhaskell/stack/pull/1583) + +Other enhancements: + +- Disable locale/codepage hacks when GHC >=7.10.3 + [#1552](https://github.com/commercialhaskell/stack/issues/1552) +- Specify multiple images to build for `stack image container` + [docs](http://docs.haskellstack.org/en/stable/yaml_configuration/#image) +- Specify which executables to include in images for `stack image container` + [docs](http://docs.haskellstack.org/en/stable/yaml_configuration/#image) +- Docker: pass supplemantary groups and umask into container +- If git fetch fails wipe the directory and try again from scratch + [#1418](https://github.com/commercialhaskell/stack/issues/1418) +- Warn if newly installed executables won't be available on the PATH + [#1362](https://github.com/commercialhaskell/stack/issues/1362) +- stack.yaml: for `stack image container`, specify multiple images to generate, + and which executables should be added to those images +- GHCI: add interactive Main selection + [#1068](https://github.com/commercialhaskell/stack/issues/1068) +- Care less about the particular name of a GHCJS sdist folder + [#1622](https://github.com/commercialhaskell/stack/issues/1622) +- Unified Enable/disable help messaging + [#1613](https://github.com/commercialhaskell/stack/issues/1613) + +Bug fixes: + +- Don't share precompiled packages between GHC/platform variants and Docker + [#1551](https://github.com/commercialhaskell/stack/issues/1551) +- Properly redownload corrupted downloads with the correct file size. + [Mailing list discussion](https://groups.google.com/d/msg/haskell-stack/iVGDG5OHYxs/FjUrR5JsDQAJ) +- Gracefully handle invalid paths in error/warning messages + [#1561](https://github.com/commercialhaskell/stack/issues/1561) +- Nix: select the correct GHC version corresponding to the snapshot + even when an abstract resolver is passed via `--resolver` on the + command-line. + [#1641](https://github.com/commercialhaskell/stack/issues/1641) +- Fix: Stack does not allow using an external package from ghci + [#1557](https://github.com/commercialhaskell/stack/issues/1557) +- Disable ambiguous global '--resolver' option for 'stack init' + [#1531](https://github.com/commercialhaskell/stack/issues/1531) +- Obey `--no-nix` flag +- Fix: GHCJS Execute.hs: Non-exhaustive patterns in lambda + [#1591](https://github.com/commercialhaskell/stack/issues/1591) +- Send file-watch and sticky logger messages to stderr + [#1302](https://github.com/commercialhaskell/stack/issues/1302) + [#1635](https://github.com/commercialhaskell/stack/issues/1635) +- Use globaldb path for querying Cabal version + [#1647](https://github.com/commercialhaskell/stack/issues/1647) + +## 1.0.0 + +Release notes: + +* We're calling this version 1.0.0 in preparation for Stackage + LTS 4. Note, however, that this does not mean the code's API + will be stable as this is primarily an end-user tool. + +Enhancements: + +* Added flag `--profile` flag: passed with `stack build`, it will + enable profiling, and for `--bench` and `--test` it will generate a + profiling report by passing `+RTS -p` to the executable(s). Great + for using like `stack build --bench --profile` (remember that + enabling profile will slow down your benchmarks by >4x). Run `stack + build --bench` again to disable the profiling and get proper speeds +* Added flag `--trace` flag: just like `--profile`, it enables + profiling, but instead of generating a report for `--bench` and + `--test`, prints out a stack trace on exception. Great for using + like `stack build --test --trace` +* Nix: all options can be overriden on command line + [#1483](https://github.com/commercialhaskell/stack/issues/1483) +* Nix: build environments (shells) are now pure by default. +* Make verbosity silent by default in script interpreter mode + [#1472](https://github.com/commercialhaskell/stack/issues/1472) +* Show a message when resetting git commit fails + [#1453](https://github.com/commercialhaskell/stack/issues/1453) +* Improve Unicode handling in project/package names + [#1337](https://github.com/commercialhaskell/stack/issues/1337) +* Fix ambiguity between a stack command and a filename to execute (prefer + `stack` subcommands) + [#1471](https://github.com/commercialhaskell/stack/issues/1471) +* Support multi line interpreter directive comments + [#1394](https://github.com/commercialhaskell/stack/issues/1394) +* Handle space separated pids in ghc-pkg dump (for GHC HEAD) + [#1509](https://github.com/commercialhaskell/stack/issues/1509) +* Add ghci --no-package-hiding option + [#1517](https://github.com/commercialhaskell/stack/issues/1517) +* `stack new` can download templates from URL + [#1466](https://github.com/commercialhaskell/stack/issues/1466) + +Bug fixes: + +* Nix: stack exec options are passed properly to the stack sub process + [#1538](https://github.com/commercialhaskell/stack/issues/1538) +* Nix: specifying a shell-file works in any current working directory + [#1547](https://github.com/commercialhaskell/stack/issues/1547) +* Nix: use `--resolver` argument +* Docker: fix missing image message and '--docker-auto-pull' +* No HTML escaping for "stack new" template params + [#1475](https://github.com/commercialhaskell/stack/issues/1475) +* Set permissions for generated .ghci script + [#1480](https://github.com/commercialhaskell/stack/issues/1480) +* Restrict commands allowed in interpreter mode + [#1504](https://github.com/commercialhaskell/stack/issues/1504) +* stack ghci doesn't see preprocessed files for executables + [#1347](https://github.com/commercialhaskell/stack/issues/1347) +* All test suites run even when only one is requested + [#1550](https://github.com/commercialhaskell/stack/pull/1550) +* Edge cases in broken templates give odd errors + [#1535](https://github.com/commercialhaskell/stack/issues/1535) +* Fix test coverage bug on windows + +## 0.1.10.1 + +Bug fixes: + +* `stack image container` did not actually build an image + [#1473](https://github.com/commercialhaskell/stack/issues/1473) + +## 0.1.10.0 + +Release notes: + +* The Stack home page is now at [haskellstack.org](http://haskellstack.org), + which shows the documentation rendered by readthedocs.org. Note: this + has necessitated some changes to the links in the documentation's markdown + source code, so please check the links on the website before submitting a PR + to fix them. +* The locations of the + [Ubuntu](http://docs.haskellstack.org/en/stable/install_and_upgrade/#ubuntu) + and + [Debian](http://docs.haskellstack.org/en/stable/install_and_upgrade/#debian) + package repositories have changed to have correct URL semantics according to + Debian's guidelines + [#1378](https://github.com/commercialhaskell/stack/issues/1378). The old + locations will continue to work for some months, but we suggest that you + adjust your `/etc/apt/sources.list.d/fpco.list` to the new location to avoid + future disruption. +* [openSUSE and SUSE Linux Enterprise](http://docs.haskellstack.org/en/stable/install_and_upgrade/#suse) + packages are now available, thanks to [@mimi1vx](https://github.com/mimi1vx). + Note: there will be some lag before these pick up new versions, as they are + based on Stackage LTS. + +Major changes: + +* Support for building inside a Nix-shell providing system dependencies + [#1285](https://github.com/commercialhaskell/stack/pull/1285) +* Add optional GPG signing on `stack upload --sign` or with + `stack sig sign ...` + +Other enhancements: + +* Print latest applicable version of packages on conflicts + [#508](https://github.com/commercialhaskell/stack/issues/508) +* Support for packages located in Mercurial repositories + [#1397](https://github.com/commercialhaskell/stack/issues/1397) +* Only run benchmarks specified as build targets + [#1412](https://github.com/commercialhaskell/stack/issues/1412) +* Support git-style executable fall-through (`stack something` executes + `stack-something` if present) + [#1433](https://github.com/commercialhaskell/stack/issues/1433) +* GHCi now loads intermediate dependencies + [#584](https://github.com/commercialhaskell/stack/issues/584) +* `--work-dir` option for overriding `.stack-work` + [#1178](https://github.com/commercialhaskell/stack/issues/1178) +* Support `detailed-0.9` tests + [#1429](https://github.com/commercialhaskell/stack/issues/1429) +* Docker: improved POSIX signal proxying to containers + [#547](https://github.com/commercialhaskell/stack/issues/547) + +Bug fixes: + +* Show absolute paths in error messages in multi-package builds + [#1348](https://github.com/commercialhaskell/stack/issues/1348) +* Docker-built binaries and libraries in different path + [#911](https://github.com/commercialhaskell/stack/issues/911) + [#1367](https://github.com/commercialhaskell/stack/issues/1367) +* Docker: `--resolver` argument didn't effect selected image tag +* GHCi: Spaces in filepaths caused module loading issues + [#1401](https://github.com/commercialhaskell/stack/issues/1401) +* GHCi: cpp-options in cabal files weren't used + [#1419](https://github.com/commercialhaskell/stack/issues/1419) +* Benchmarks couldn't be run independently of eachother + [#1412](https://github.com/commercialhaskell/stack/issues/1412) +* Send output of building setup to stderr + [#1410](https://github.com/commercialhaskell/stack/issues/1410) + +## 0.1.8.0 + +Major changes: + +* GHCJS can now be used with stackage snapshots via the new `compiler` field. +* Windows installers are now available: + [download them here](http://docs.haskellstack.org/en/stable/install_and_upgrade/#windows) + [#613](https://github.com/commercialhaskell/stack/issues/613) +* Docker integration works with non-FPComplete generated images + [#531](https://github.com/commercialhaskell/stack/issues/531) + +Other enhancements: + +* Added an `allow-newer` config option + [#922](https://github.com/commercialhaskell/stack/issues/922) + [#770](https://github.com/commercialhaskell/stack/issues/770) +* When a Hackage revision invalidates a build plan in a snapshot, trust the + snapshot [#770](https://github.com/commercialhaskell/stack/issues/770) +* Added a `stack config set resolver RESOLVER` command. Part of work on + [#115](https://github.com/commercialhaskell/stack/issues/115) +* `stack setup` can now install GHCJS on windows. See + [#1145](https://github.com/commercialhaskell/stack/issues/1145) and + [#749](https://github.com/commercialhaskell/stack/issues/749) +* `stack hpc report` command added, which generates reports for HPC tix files +* `stack ghci` now accepts all the flags accepted by `stack build`. See + [#1186](https://github.com/commercialhaskell/stack/issues/1186) +* `stack ghci` builds the project before launching GHCi. If the build fails, + optimistically launch GHCi anyway. Use `stack ghci --no-build` option to + disable [#1065](https://github.com/commercialhaskell/stack/issues/1065) +* `stack ghci` now detects and warns about various circumstances where it is + liable to fail. See + [#1270](https://github.com/commercialhaskell/stack/issues/1270) +* Added `require-docker-version` configuration option +* Packages will now usually be built along with their tests and benchmarks. See + [#1166](https://github.com/commercialhaskell/stack/issues/1166) +* Relative `local-bin-path` paths will be relative to the project's root + directory, not the current working directory. + [#1340](https://github.com/commercialhaskell/stack/issues/1340) +* `stack clean` now takes an optional `[PACKAGE]` argument for use in + multi-package projects. See + [#583](https://github.com/commercialhaskell/stack/issues/583) +* Ignore cabal_macros.h as a dependency + [#1195](https://github.com/commercialhaskell/stack/issues/1195) +* Pad timestamps and show local time in --verbose output + [#1226](https://github.com/commercialhaskell/stack/issues/1226) +* GHCi: Import all modules after loading them + [#995](https://github.com/commercialhaskell/stack/issues/995) +* Add subcommand aliases: `repl` for `ghci`, and `runhaskell` for `runghc` + [#1241](https://github.com/commercialhaskell/stack/issues/1241) +* Add typo recommendations for unknown package identifiers + [#158](https://github.com/commercialhaskell/stack/issues/158) +* Add `stack path --local-hpc-root` option +* Overhaul dependencies' haddocks copying + [#1231](https://github.com/commercialhaskell/stack/issues/1231) +* Support for extra-package-dbs in 'stack ghci' + [#1229](https://github.com/commercialhaskell/stack/pull/1229) +* `stack new` disallows package names with "words" consisting solely of numbers + [#1336](https://github.com/commercialhaskell/stack/issues/1336) +* `stack build --fast` turns off optimizations +* Show progress while downloading package index + [#1223](https://github.com/commercialhaskell/stack/issues/1223). + +Bug fixes: + +* Fix: Haddocks not copied for dependencies + [#1105](https://github.com/commercialhaskell/stack/issues/1105) +* Fix: Global options did not work consistently after subcommand + [#519](https://github.com/commercialhaskell/stack/issues/519) +* Fix: 'stack ghci' doesn't notice that a module got deleted + [#1180](https://github.com/commercialhaskell/stack/issues/1180) +* Rebuild when cabal file is changed +* Fix: Paths in GHC warnings not canonicalized, nor those for packages in + subdirectories or outside the project root + [#1259](https://github.com/commercialhaskell/stack/issues/1259) +* Fix: unlisted files in tests and benchmarks trigger extraneous second build + [#838](https://github.com/commercialhaskell/stack/issues/838) + +## 0.1.6.0 + +Major changes: + +* `stack setup` now supports building and booting GHCJS from source tarball. +* On Windows, build directories no longer display "pretty" information + (like x86_64-windows/Cabal-1.22.4.0), but rather a hash of that + content. The reason is to avoid the 260 character path limitation on + Windows. See + [#1027](https://github.com/commercialhaskell/stack/pull/1027) +* Rename config files and clarify their purposes [#969](https://github.com/commercialhaskell/stack/issues/969) + * `~/.stack/stack.yaml` --> `~/.stack/config.yaml` + * `~/.stack/global` --> `~/.stack/global-project` + * `/etc/stack/config` --> `/etc/stack/config.yaml` + * Old locations still supported, with deprecation warnings +* New command "stack eval CODE", which evaluates to "stack exec ghc -- -e CODE". + +Other enhancements: + +* No longer install `git` on Windows + [#1046](https://github.com/commercialhaskell/stack/issues/1046). You + can still get this behavior by running the following yourself: + `stack exec -- pacman -Sy --noconfirm git`. +* Typing enter during --file-watch triggers a rebuild [#1023](https://github.com/commercialhaskell/stack/pull/1023) +* Use Haddock's `--hyperlinked-source` (crosslinked source), if available [#1070](https://github.com/commercialhaskell/stack/pull/1070) +* Use Stack-installed GHCs for `stack init --solver` [#1072](https://github.com/commercialhaskell/stack/issues/1072) +* New experimental `stack query` command [#1087](https://github.com/commercialhaskell/stack/issues/1087) +* By default, stack no longer rebuilds a package due to GHC options changes. This behavior can be tweaked with the `rebuild-ghc-options` setting. [#1089](https://github.com/commercialhaskell/stack/issues/1089) +* By default, ghc-options are applied to all local packages, not just targets. This behavior can be tweaked with the `apply-ghc-options` setting. [#1089](https://github.com/commercialhaskell/stack/issues/1089) +* Docker: download or override location of stack executable to re-run in container [#974](https://github.com/commercialhaskell/stack/issues/974) +* Docker: when Docker Engine is remote, don't run containerized processes as host's UID/GID [#194](https://github.com/commercialhaskell/stack/issues/194) +* Docker: `set-user` option to enable/disable running containerized processes as host's UID/GID [#194](https://github.com/commercialhaskell/stack/issues/194) +* Custom Setup.hs files are now precompiled instead of interpreted. This should be a major performance win for certain edge cases (biggest example: [building Cabal itself](https://github.com/commercialhaskell/stack/issues/1041)) while being either neutral or a minor slowdown for more common cases. +* `stack test --coverage` now also generates a unified coverage report for multiple test-suites / packages. In the unified report, test-suites can contribute to the coverage of other packages. + +Bug fixes: + +* Ignore stack-built executables named `ghc` + [#1052](https://github.com/commercialhaskell/stack/issues/1052) +* Fix quoting of output failed command line arguments +* Mark executable-only packages as installed when copied from cache [#1043](https://github.com/commercialhaskell/stack/pull/1043) +* Canonicalize temporary directory paths [#1047](https://github.com/commercialhaskell/stack/pull/1047) +* Put code page fix inside the build function itself [#1066](https://github.com/commercialhaskell/stack/issues/1066) +* Add `explicit-setup-deps` option [#1110](https://github.com/commercialhaskell/stack/issues/1110), and change the default to the old behavior of using any package in the global and snapshot database [#1025](https://github.com/commercialhaskell/stack/issues/1025) +* Precompiled cache checks full package IDs on Cabal < 1.22 [#1103](https://github.com/commercialhaskell/stack/issues/1103) +* Pass -package-id to ghci [#867](https://github.com/commercialhaskell/stack/issues/867) +* Ignore global packages when copying precompiled packages [#1146](https://github.com/commercialhaskell/stack/issues/1146) + +## 0.1.5.0 + +Major changes: + +* On Windows, we now use a full MSYS2 installation in place of the previous PortableGit. This gives you access to the pacman package manager for more easily installing libraries. +* Support for custom GHC binary distributions [#530](https://github.com/commercialhaskell/stack/issues/530) + * `ghc-variant` option in stack.yaml to specify the variant (also + `--ghc-variant` command-line option) + * `setup-info` in stack.yaml, to specify where to download custom binary + distributions (also `--ghc-bindist` command-line option) + * Note: On systems with libgmp4 (aka `libgmp.so.3`), such as CentOS 6, you + may need to re-run `stack setup` due to the centos6 GHC bindist being + treated like a variant +* A new `--pvp-bounds` flag to the sdist and upload commands allows automatic adding of PVP upper and/or lower bounds to your dependencies + +Other enhancements: + +* Adapt to upcoming Cabal installed package identifier format change [#851](https://github.com/commercialhaskell/stack/issues/851) +* `stack setup` takes a `--stack-setup-yaml` argument +* `--file-watch` is more discerning about which files to rebuild for [#912](https://github.com/commercialhaskell/stack/issues/912) +* `stack path` now supports `--global-pkg-db` and `--ghc-package-path` +* `--reconfigure` flag [#914](https://github.com/commercialhaskell/stack/issues/914) [#946](https://github.com/commercialhaskell/stack/issues/946) +* Cached data is written with a checksum of its structure [#889](https://github.com/commercialhaskell/stack/issues/889) +* Fully removed `--optimizations` flag +* Added `--cabal-verbose` flag +* Added `--file-watch-poll` flag for polling instead of using filesystem events (useful for running tests in a Docker container while modifying code in the host environment. When code is injected into the container via a volume, the container won't propagate filesystem events). +* Give a preemptive error message when `-prof` is given as a GHC option [#1015](https://github.com/commercialhaskell/stack/issues/1015) +* Locking is now optional, and will be turned on by setting the `STACK_LOCK` environment variable to `true` [#950](https://github.com/commercialhaskell/stack/issues/950) +* Create default stack.yaml with documentation comments and commented out options [#226](https://github.com/commercialhaskell/stack/issues/226) +* Out of memory warning if Cabal exits with -9 [#947](https://github.com/commercialhaskell/stack/issues/947) + +Bug fixes: + +* Hacky workaround for optparse-applicative issue with `stack exec --help` [#806](https://github.com/commercialhaskell/stack/issues/806) +* Build executables for local extra deps [#920](https://github.com/commercialhaskell/stack/issues/920) +* copyFile can't handle directories [#942](https://github.com/commercialhaskell/stack/pull/942) +* Support for spaces in Haddock interface files [fpco/minghc#85](https://github.com/fpco/minghc/issues/85) +* Temporarily building against a "shadowing" local package? [#992](https://github.com/commercialhaskell/stack/issues/992) +* Fix Setup.exe name for --upgrade-cabal on Windows [#1002](https://github.com/commercialhaskell/stack/issues/1002) +* Unlisted dependencies no longer trigger extraneous second build [#838](https://github.com/commercialhaskell/stack/issues/838) + +## 0.1.4.1 + +Fix stack's own Haddocks. No changes to functionality (only comments updated). + +## 0.1.4.0 + +Major changes: + +* You now have more control over how GHC versions are matched, e.g. "use exactly this version," "use the specified minor version, but allow patches," or "use the given minor version or any later minor in the given major release." The default has switched from allowing newer later minor versions to a specific minor version allowing patches. For more information, see [#736](https://github.com/commercialhaskell/stack/issues/736) and [#784](https://github.com/commercialhaskell/stack/pull/784). +* Support added for compiling with GHCJS +* stack can now reuse prebuilt binaries between snapshots. That means that, if you build package foo in LTS-3.1, that binary version can be reused in LTS-3.2, assuming it uses the same dependencies and flags. [#878](https://github.com/commercialhaskell/stack/issues/878) + +Other enhancements: + +* Added the `--docker-env` argument, to set environment variables in Docker container. +* Set locale environment variables to UTF-8 encoding for builds to avoid "commitBuffer: invalid argument" errors from GHC [#793](https://github.com/commercialhaskell/stack/issues/793) +* Enable translitation for encoding on stdout and stderr [#824](https://github.com/commercialhaskell/stack/issues/824) +* By default, `stack upgrade` automatically installs GHC as necessary [#797](https://github.com/commercialhaskell/stack/issues/797) +* Added the `ghc-options` field to stack.yaml [#796](https://github.com/commercialhaskell/stack/issues/796) +* Added the `extra-path` field to stack.yaml +* Code page changes on Windows only apply to the build command (and its synonyms), and can be controlled via a command line flag (still defaults to on) [#757](https://github.com/commercialhaskell/stack/issues/757) +* Implicitly add packages to extra-deps when a flag for them is set [#807](https://github.com/commercialhaskell/stack/issues/807) +* Use a precompiled Setup.hs for simple build types [#801](https://github.com/commercialhaskell/stack/issues/801) +* Set --enable-tests and --enable-benchmarks optimistically [#805](https://github.com/commercialhaskell/stack/issues/805) +* `--only-configure` option added [#820](https://github.com/commercialhaskell/stack/issues/820) +* Check for duplicate local package names +* Stop nagging people that call `stack test` [#845](https://github.com/commercialhaskell/stack/issues/845) +* `--file-watch` will ignore files that are in your VCS boring/ignore files [#703](https://github.com/commercialhaskell/stack/issues/703) +* Add `--numeric-version` option + +Bug fixes: + +* `stack init --solver` fails if `GHC_PACKAGE_PATH` is present [#860](https://github.com/commercialhaskell/stack/issues/860) +* `stack solver` and `stack init --solver` check for test suite and benchmark dependencies [#862](https://github.com/commercialhaskell/stack/issues/862) +* More intelligent logic for setting UTF-8 locale environment variables [#856](https://github.com/commercialhaskell/stack/issues/856) +* Create missing directories for `stack sdist` +* Don't ignore .cabal files with extra periods [#895](https://github.com/commercialhaskell/stack/issues/895) +* Deprecate unused `--optimizations` flag +* Truncated output on slow terminals [#413](https://github.com/commercialhaskell/stack/issues/413) + +## 0.1.3.1 + +Bug fixes: + +* Ignore disabled executables [#763](https://github.com/commercialhaskell/stack/issues/763) + +## 0.1.3.0 + +Major changes: + +* Detect when a module is compiled but not listed in the cabal file ([#32](https://github.com/commercialhaskell/stack/issues/32)) + * A warning is displayed for any modules that should be added to `other-modules` in the .cabal file + * These modules are taken into account when determining whether a package needs to be built +* Respect TemplateHaskell addDependentFile dependency changes ([#105](https://github.com/commercialhaskell/stack/issues/105)) + * TH dependent files are taken into account when determining whether a package needs to be built. +* Overhauled target parsing, added `--test` and `--bench` options [#651](https://github.com/commercialhaskell/stack/issues/651) + * For details, see [Build commands documentation](http://docs.haskellstack.org/en/stable/build_command/) + +Other enhancements: + +* Set the `HASKELL_DIST_DIR` environment variable [#524](https://github.com/commercialhaskell/stack/pull/524) +* Track build status of tests and benchmarks [#525](https://github.com/commercialhaskell/stack/issues/525) +* `--no-run-tests` [#517](https://github.com/commercialhaskell/stack/pull/517) +* Targets outside of root dir don't build [#366](https://github.com/commercialhaskell/stack/issues/366) +* Upper limit on number of flag combinations to test [#543](https://github.com/commercialhaskell/stack/issues/543) +* Fuzzy matching support to give better error messages for close version numbers [#504](https://github.com/commercialhaskell/stack/issues/504) +* `--local-bin-path` global option. Use to change where binaries get placed on a `--copy-bins` [#342](https://github.com/commercialhaskell/stack/issues/342) +* Custom snapshots [#111](https://github.com/commercialhaskell/stack/issues/111) +* --force-dirty flag: Force treating all local packages as having dirty files (useful for cases where stack can't detect a file change) +* GHC error messages: display file paths as absolute instead of relative for better editor integration +* Add the `--copy-bins` option [#569](https://github.com/commercialhaskell/stack/issues/569) +* Give warnings on unexpected config keys [#48](https://github.com/commercialhaskell/stack/issues/48) +* Remove Docker `pass-host` option +* Don't require cabal-install to upload [#313](https://github.com/commercialhaskell/stack/issues/313) +* Generate indexes for all deps and all installed snapshot packages [#143](https://github.com/commercialhaskell/stack/issues/143) +* Provide `--resolver global` option [#645](https://github.com/commercialhaskell/stack/issues/645) + * Also supports `--resolver nightly`, `--resolver lts`, and `--resolver lts-X` +* Make `stack build --flag` error when flag or package is unknown [#617](https://github.com/commercialhaskell/stack/issues/617) +* Preserve file permissions when unpacking sources [#666](https://github.com/commercialhaskell/stack/pull/666) +* `stack build` etc work outside of a project +* `list-dependencies` command [#638](https://github.com/commercialhaskell/stack/issues/638) +* `--upgrade-cabal` option to `stack setup` [#174](https://github.com/commercialhaskell/stack/issues/174) +* `--exec` option [#651](https://github.com/commercialhaskell/stack/issues/651) +* `--only-dependencies` implemented correctly [#387](https://github.com/commercialhaskell/stack/issues/387) + +Bug fixes: + +* Extensions from the `other-extensions` field no longer enabled by default [#449](https://github.com/commercialhaskell/stack/issues/449) +* Fix: haddock forces rebuild of empty packages [#452](https://github.com/commercialhaskell/stack/issues/452) +* Don't copy over executables excluded by component selection [#605](https://github.com/commercialhaskell/stack/issues/605) +* Fix: stack fails on Windows with git package in stack.yaml and no git binary on path [#712](https://github.com/commercialhaskell/stack/issues/712) +* Fixed GHCi issue: Specifying explicit package versions (#678) +* Fixed GHCi issue: Specifying -odir and -hidir as .stack-work/odir (#529) +* Fixed GHCi issue: Specifying A instead of A.ext for modules (#498) + +## 0.1.2.0 + +* Add `--prune` flag to `stack dot` [#487](https://github.com/commercialhaskell/stack/issues/487) +* Add `--[no-]external`,`--[no-]include-base` flags to `stack dot` [#437](https://github.com/commercialhaskell/stack/issues/437) +* Add `--ignore-subdirs` flag to init command [#435](https://github.com/commercialhaskell/stack/pull/435) +* Handle attempt to use non-existing resolver [#436](https://github.com/commercialhaskell/stack/pull/436) +* Add `--force` flag to `init` command +* exec style commands accept the `--package` option (see [Reddit discussion](http://www.reddit.com/r/haskell/comments/3bd66h/stack_runghc_turtle_as_haskell_script_solution/)) +* `stack upload` without arguments doesn't do anything [#439](https://github.com/commercialhaskell/stack/issues/439) +* Print latest version of packages on conflicts [#450](https://github.com/commercialhaskell/stack/issues/450) +* Flag to avoid rerunning tests that haven't changed [#451](https://github.com/commercialhaskell/stack/issues/451) +* stack can act as a script interpreter (see [Script interpreter] (https://github.com/commercialhaskell/stack/wiki/Script-interpreter) and [Reddit discussion](http://www.reddit.com/r/haskell/comments/3bd66h/stack_runghc_turtle_as_haskell_script_solution/)) +* Add the __`--file-watch`__ flag to auto-rebuild on file changes [#113](https://github.com/commercialhaskell/stack/issues/113) +* Rename `stack docker exec` to `stack exec --plain` +* Add the `--skip-msys` flag [#377](https://github.com/commercialhaskell/stack/issues/377) +* `--keep-going`, turned on by default for tests and benchmarks [#478](https://github.com/commercialhaskell/stack/issues/478) +* `concurrent-tests: BOOL` [#492](https://github.com/commercialhaskell/stack/issues/492) +* Use hashes to check file dirtiness [#502](https://github.com/commercialhaskell/stack/issues/502) +* Install correct GHC build on systems with libgmp.so.3 [#465](https://github.com/commercialhaskell/stack/issues/465) +* `stack upgrade` checks version before upgrading [#447](https://github.com/commercialhaskell/stack/issues/447) + +## 0.1.1.0 + +* Remove GHC uncompressed tar file after installation [#376](https://github.com/commercialhaskell/stack/issues/376) +* Put stackage snapshots JSON on S3 [#380](https://github.com/commercialhaskell/stack/issues/380) +* Specifying flags for multiple packages [#335](https://github.com/commercialhaskell/stack/issues/335) +* single test suite failure should show entire log [#388](https://github.com/commercialhaskell/stack/issues/388) +* valid-wanted is a confusing option name [#386](https://github.com/commercialhaskell/stack/issues/386) +* stack init in multi-package project should use local packages for dependency checking [#384](https://github.com/commercialhaskell/stack/issues/384) +* Display information on why a snapshot was rejected [#381](https://github.com/commercialhaskell/stack/issues/381) +* Give a reason for unregistering packages [#389](https://github.com/commercialhaskell/stack/issues/389) +* `stack exec` accepts the `--no-ghc-package-path` parameter +* Don't require build plan to upload [#400](https://github.com/commercialhaskell/stack/issues/400) +* Specifying test components only builds/runs those tests [#398](https://github.com/commercialhaskell/stack/issues/398) +* `STACK_EXE` environment variable +* Add the `stack dot` command +* `stack upgrade` added [#237](https://github.com/commercialhaskell/stack/issues/237) +* `--stack-yaml` command line flag [#378](https://github.com/commercialhaskell/stack/issues/378) +* `--skip-ghc-check` command line flag [#423](https://github.com/commercialhaskell/stack/issues/423) + +Bug fixes: + +* Haddock links to global packages no longer broken on Windows [#375](https://github.com/commercialhaskell/stack/issues/375) +* Make flags case-insensitive [#397](https://github.com/commercialhaskell/stack/issues/397) +* Mark packages uninstalled before rebuilding [#365](https://github.com/commercialhaskell/stack/issues/365) + +## 0.1.0.0 + +* Fall back to cabal dependency solver when a snapshot can't be found +* Basic implementation of `stack new` [#137](https://github.com/commercialhaskell/stack/issues/137) +* `stack solver` command [#364](https://github.com/commercialhaskell/stack/issues/364) +* `stack path` command [#95](https://github.com/commercialhaskell/stack/issues/95) +* Haddocks [#143](https://github.com/commercialhaskell/stack/issues/143): + * Build for dependencies + * Use relative links + * Generate module contents and index for all packages in project + +## 0.0.3 + +* `--prefetch` [#297](https://github.com/commercialhaskell/stack/issues/297) +* `upload` command ported from stackage-upload [#225](https://github.com/commercialhaskell/stack/issues/225) +* `--only-snapshot` [#310](https://github.com/commercialhaskell/stack/issues/310) +* `--resolver` [#224](https://github.com/commercialhaskell/stack/issues/224) +* `stack init` [#253](https://github.com/commercialhaskell/stack/issues/253) +* `--extra-include-dirs` and `--extra-lib-dirs` [#333](https://github.com/commercialhaskell/stack/issues/333) +* Specify intra-package target [#201](https://github.com/commercialhaskell/stack/issues/201) + +## 0.0.2 + +* Fix some Windows specific bugs [#216](https://github.com/commercialhaskell/stack/issues/216) +* Improve output for package index updates [#227](https://github.com/commercialhaskell/stack/issues/227) +* Automatically update indices as necessary [#227](https://github.com/commercialhaskell/stack/issues/227) +* --verbose flag [#217](https://github.com/commercialhaskell/stack/issues/217) +* Remove packages (HTTPS and Git) [#199](https://github.com/commercialhaskell/stack/issues/199) +* Config values for system-ghc and install-ghc +* Merge `stack deps` functionality into `stack build` +* `install` command [#153](https://github.com/commercialhaskell/stack/issues/153) and [#272](https://github.com/commercialhaskell/stack/issues/272) +* overriding architecture value (useful to force 64-bit GHC on Windows, for example) +* Overhauled test running (allows cycles, avoids unnecessary recompilation, etc) + +## 0.0.1 + +* First public release, beta quality diff --git a/doc/GUIDE.md b/doc/GUIDE.md new file mode 100644 --- /dev/null +++ b/doc/GUIDE.md @@ -0,0 +1,2183 @@ +# User guide + +stack is a modern, cross-platform build tool for Haskell code. + +This guide takes a new stack user through the typical workflows. This guide +will not teach Haskell or involve much code, and it requires no prior experience +with the Haskell packaging system or other build tools. + +## Stack's functions + +stack handles the management of your toolchain (including GHC — the Glasgow +Haskell Compiler — and, for Windows users, MSYS), building and registering +libraries, building build tool dependencies, and more. While it can use existing +tools on your system, stack has the capacity to be your one-stop shop for all +Haskell tooling you need. This guide will follow that stack-centric approach. + +### What makes stack special? + +The primary stack design point is __reproducible builds__. If you run `stack +build` today, you should get the same result running `stack build` tomorrow. +There are some cases that can break that rule (changes in your operating system +configuration, for example), but, overall, stack follows this design philosophy +closely. To make this a simple process, stack uses curated package sets +called __snapshots__. + +stack has also been designed from the ground up to be user friendly, with an +intuitive, discoverable command line interface. For many users, simply +downloading stack and reading `stack --help` will be enough to get up and +running. This guide provides a more gradual tour for users who prefer that +learning style. + +To build your project, stack uses a `stack.yaml` file in the root directory of +your project as a sort of blueprint. That file contains a reference, called a +__resolver__, to the snapshot which your package will be built against. + +Finally, stack is __isolated__: it will not make changes outside of specific +stack directories. stack-built files generally go in either the stack root +directory (default `~/.stack`) or `./.stack-work` directories local to each +project. The stack root directory holds packages belonging to snapshots and any +stack-installed versions of GHC. Stack will not tamper with any system version +of GHC or interfere with packages installed by `cabal` or any other build tools. + +_NOTE_ In this guide, we'll use commands as run on a GNU/Linux system +(specifically Ubuntu 14.04, 64-bit) and share output from that. Output on other +systems — or with different versions of stack — will be slightly different, but +all commands work cross-platform, unless explicitly stated otherwise. + +## Downloading and Installation + +The [documentation dedicated to downloading +stack](install_and_upgrade.md) has the most +up-to-date information for a variety of operating systems, including multiple +GNU/Linux flavors. Instead of repeating that content here, please go check out +that page and come back here when you can successfully run `stack --version`. +The rest of this session will demonstrate the installation procedure on a +vanilla Ubuntu 14.04 machine. + +``` +# Starting with a *really* bare machine +michael@d30748af6d3d:~$ sudo apt-get install wget +# Demonstrate that stack really isn't available +michael@d30748af6d3d:~$ stack +-bash: stack: command not found +# Get the signing key for the package repo +michael@d30748af6d3d:~$ wget -q -O- https://s3.amazonaws.com/download.fpcomplete.com/ubuntu/fpco.key | sudo apt-key add - +OK +michael@d30748af6d3d:~$ echo 'deb http://download.fpcomplete.com/ubuntu/trusty stable main'|sudo tee /etc/apt/sources.list.d/fpco.list +deb http://download.fpcomplete.com/ubuntu/trusty stable main +michael@d30748af6d3d:~$ sudo apt-get update && sudo apt-get install stack -y +# downloading... +michael@d30748af6d3d:~$ stack --version +Version 0.1.3.1, Git revision 908b04205e6f436d4a5f420b1c6c646ed2b804d7 +``` + +With stack now up and running, you're good to go. Though not required, we +recommend setting your PATH environment variable to include `$HOME/.local/bin`: + +``` +michael@d30748af6d3d:~$ echo 'export PATH=$HOME/.local/bin:$PATH' >> ~/.bashrc +``` + +## Hello World Example + +With stack installed, let's create a new project from a template and walk +through the most common stack commands. + +### stack new + +We'll start off with the `stack new` command to create a new +*project*. We'll call our project `helloworld`, and we'll use the +`new-template` project template: + +``` +michael@d30748af6d3d:~$ stack new helloworld new-template +``` + +For this first stack command, there's quite a bit of initial setup it needs to +do (such as downloading the list of packages available upstream), so you'll see +a lot of output. Though your exact results may vary, below is an example of the +sort of output you will see. Over the course of this guide a lot of the content +will begin to make more sense: + +``` +Downloading template "new-template" to create project "helloworld" in helloworld/ ... +Using the following authorship configuration: +author-email: example@example.com +author-name: Example Author Name +Copy these to /home/michael/.stack/config.yaml and edit to use different values. +Writing default config file to: /home/michael/helloworld/stack.yaml +Basing on cabal files: +- /home/michael/helloworld/helloworld.cabal + +Downloaded lts-3.2 build plan. +Caching build plan +Fetched package index. +Populated index cache. +Checking against build plan lts-3.2 +Selected resolver: lts-3.2 +Wrote project config to: /home/michael/helloworld/stack.yaml +``` +We now have a project in the `helloworld` directory! + +### stack setup + +Instead of assuming you want stack to download and install +GHC for you, it asks you to do this as a separate command: +`setup`. If we don't run `stack setup` now, we'll later see a +message that we are missing the right GHC version. + +Let's run stack setup: + +``` +michael@d30748af6d3d:~/helloworld$ stack setup +Downloaded ghc-7.10.2. +Installed GHC. +stack will use a locally installed GHC +For more information on paths, see 'stack path' and 'stack exec env' +To use this GHC and packages outside of a project, consider using: +stack ghc, stack ghci, stack runghc, or stack exec +``` + +It doesn't come through in the output here, but you'll get intermediate +download percentage statistics while the download is occurring. This command +may take some time, depending on download speeds. + +__NOTE__: GHC will be installed to your global stack root directory, so +calling `ghc` on the command line won't work. See the `stack exec`, +`stack ghc`, and `stack runghc` commands below for more information. + +### stack build + +Next, we'll run the most important stack command: `stack build`. + +__NOTE__: If you forgot to run `stack setup` in the previous step you'll get an +error: + +``` +michael@d30748af6d3d:~$ cd helloworld/ +michael@d30748af6d3d:~/helloworld$ stack build +No GHC found, expected version 7.10.2 (x86_64) (based on resolver setting in /home/michael/helloworld/stack.yaml). +Try running stack setup +``` + +stack needs GHC in order to build your project, and `stack setup` must be run to +check whether GHC is available (and install it if not). + +Having run `stack setup` successfully, `stack build` should build our project: + +``` +michael@d30748af6d3d:~/helloworld$ stack build +helloworld-0.1.0.0: configure +Configuring helloworld-0.1.0.0... +helloworld-0.1.0.0: build +Preprocessing library helloworld-0.1.0.0... +[1 of 1] Compiling Lib ( src/Lib.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/Lib.o ) +In-place registering helloworld-0.1.0.0... +Preprocessing executable 'helloworld-exe' for helloworld-0.1.0.0... +[1 of 1] Compiling Main ( app/Main.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/helloworld-exe/helloworld-exe-tmp/Main.o ) +Linking .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/helloworld-exe/helloworld-exe ... +helloworld-0.1.0.0: install +Installing library in +/home/michael/helloworld/.stack-work/install/x86_64-linux/lts-3.2/7.10.2/lib/x86_64-linux-ghc-7.10.2/helloworld-0.1.0.0-6urpPe0MO7OHasGCFSyIAT +Installing executable(s) in +/home/michael/helloworld/.stack-work/install/x86_64-linux/lts-3.2/7.10.2/bin +Registering helloworld-0.1.0.0... +``` + +### stack exec + +Looking closely at the output of the previous command, you can see that it built +both a library called "helloworld" and an executable called "helloworld-exe". +We'll explain more in the next section, but, for now, just notice that the +executables are installed in our project's `./stack-work` directory. + +Now, Let's use `stack exec` to run our executable (which just outputs the string +"someFunc"): + +``` +michael@d30748af6d3d:~/helloworld$ stack exec helloworld-exe +someFunc +``` + +`stack exec` works by providing the same reproducible environment that was used +to build your project to the command that you are running. Thus, it knew where +to find `helloworld-exe` even though it is hidden in the `./stack-work` +directory. + +### stack test + +Finally, like all good software, helloworld actually has a test suite. +Let's run it with `stack test`: + +``` +michael@d30748af6d3d:~/helloworld$ stack test +NOTE: the test command is functionally equivalent to 'build --test' +helloworld-0.1.0.0: configure (test) +Configuring helloworld-0.1.0.0... +helloworld-0.1.0.0: build (test) +Preprocessing library helloworld-0.1.0.0... +In-place registering helloworld-0.1.0.0... +Preprocessing test suite 'helloworld-test' for helloworld-0.1.0.0... +[1 of 1] Compiling Main ( test/Spec.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/helloworld-test/helloworld-test-tmp/Main.o ) +Linking .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/helloworld-test/helloworld-test ... +helloworld-0.1.0.0: test (suite: helloworld-test) +Test suite not yet implemented +``` + +Reading the output, you'll see that stack first builds the test suite and then +automatically runs it for us. For both the `build` and `test` command, already +built components are not built again. You can see this by running `stack build` +and `stack test` a second time: + +``` +michael@d30748af6d3d:~/helloworld$ stack build +michael@d30748af6d3d:~/helloworld$ stack test +NOTE: the test command is functionally equivalent to 'build --test' +helloworld-0.1.0.0: test (suite: helloworld-test) +Test suite not yet implemented +``` + +## Inner Workings of stack + +In this subsection, we'll dissect the helloworld example in more detail. + +### Files in helloworld + +Before studying stack more, let's understand our project a bit better. + +``` +michael@d30748af6d3d:~/helloworld$ find * -type f +LICENSE +Setup.hs +app/Main.hs +helloworld.cabal +src/Lib.hs +stack.yaml +test/Spec.hs +``` + +The `app/Main.hs`, `src/Lib.hs`, and `test/Spec.hs` files are all Haskell source +files that compose the actual functionality of our project (we won't dwell on +them here). The LICENSE file has no impact on the build, but is there for +informational/legal purposes only. The files of interest here are Setup.hs, +helloworld.cabal, and stack.yaml. + +The Setup.hs file is a component of the Cabal build system which stack uses. +It's technically not needed by stack, but it is still considered good practice +in the Haskell world to include it. The file we're using is straight +boilerplate: + +```haskell +import Distribution.Simple +main = defaultMain +``` + +Next, let's look at our stack.yaml file, which gives our project-level settings: + +```yaml +flags: {} +packages: +- '.' +extra-deps: [] +resolver: lts-3.2 +``` + +If you're familiar with YAML, you may recognize that the `flags` and +`extra-deps` keys have empty values. We'll see more interesting usages for these +fields later. Let's focus on the other two fields. `packages` tells stack which +local packages to build. In our simple example, we have only a single package in +our project, located in the same directory, so `'.'` suffices. However, stack +has powerful support for multi-package projects, which we'll elaborate on as +this guide progresses. + +The final field is resolver. This tells stack *how* to build your package: +which GHC version to use, versions of package dependencies, and so on. Our +value here says to use [LTS Haskell version +3.2](https://www.stackage.org/lts-3.2), which implies GHC 7.10.2 (which is why +`stack setup` installs that version of GHC). There are a number of values you +can use for resolver, which we'll cover later. + +The final file of import is helloworld.cabal. stack is built on top of the +Cabal build system. In Cabal, we have individual *packages*, each of which +contains a single .cabal file. The .cabal file can define 1 or more +*components*: a library, executables, test suites, and benchmarks. It also +specifies additional information such as library dependencies, default language +pragmas, and so on. + +In this guide, we'll discuss the bare minimum necessary to understand how to +modify a .cabal file. Haskell.org has the definitive [reference for the .cabal +file format](https://www.haskell.org/cabal/users-guide/developing-packages.html). + +### The setup command + +As we saw above, the `setup` command installed GHC for us. Just for kicks, +let's run `setup` a second time: + +``` +michael@d30748af6d3d:~/helloworld$ stack setup +stack will use a locally installed GHC +For more information on paths, see 'stack path' and 'stack exec env' +To use this GHC and packages outside of a project, consider using: +stack ghc, stack ghci, stack runghc, or stack exec +``` + +Thankfully, the command is smart enough to know not to perform an installation +twice. `setup` will either use the first GHC it finds on your PATH, or a locally +installed version. As the command output above indicates, you can use `stack +path` for quite a bit of path information (which we'll play with more later). +For now, we'll just look at where GHC is installed: + +``` +michael@d30748af6d3d:~/helloworld$ stack exec which ghc +/home/michael/.stack/programs/x86_64-linux/ghc-7.10.2/bin/ghc +``` + +As you can see from that path (and as emphasized earlier), the installation is +placed to not interfere with any other GHC installation, whether system-wide or +even different GHC versions installed by stack. + +### The build command + +The build command is the heart and soul of stack. It is the engine that powers +building your code, testing it, getting dependencies, and more. Quite a bit of +the remainder of this guide will cover more advanced `build` functions and +features, such as building test and Haddocks at the same time, or constantly +rebuilding blocking on file changes. + +*On a philosophical note:* Running the build command twice with the same +options and arguments should generally be a no-op (besides things like +rerunning test suites), and should, in general, produce a reproducible result +between different runs. + +## Adding dependencies + +Let's say we decide to modify our helloworld source a bit to use a new library, +perhaps the ubiquitous text package. For example: + +```haskell +{-# LANGUAGE OverloadedStrings #-} +module Lib + ( someFunc + ) where + +import qualified Data.Text.IO as T + +someFunc :: IO () +someFunc = T.putStrLn "someFunc" +``` + +When we try to build this, things don't go as expected: + +```haskell +michael@d30748af6d3d:~/helloworld$ stack build +helloworld-0.1.0.0-c91e853ce4bfbf6d394f54b135573db8: unregistering (local file changes) +helloworld-0.1.0.0: configure +Configuring helloworld-0.1.0.0... +helloworld-0.1.0.0: build +Preprocessing library helloworld-0.1.0.0... + +/home/michael/helloworld/src/Lib.hs:6:18: + Could not find module `Data.Text.IO' + Use -v to see a list of the files searched for. + +-- While building package helloworld-0.1.0.0 using: + /home/michael/.stack/programs/x86_64-linux/ghc-7.10.2/bin/runhaskell -package=Cabal-1.22.4.0 -clear-package-db -global-package-db -package-db=/home/michael/.stack/snapshots/x86_64-linux/lts-3.2/7.10.2/pkgdb/ /tmp/stack5846/Setup.hs --builddir=.stack-work/dist/x86_64-linux/Cabal-1.22.4.0/ build exe:helloworld-exe --ghc-options -hpcdir .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/hpc/.hpc/ -ddump-hi -ddump-to-file + Process exited with code: ExitFailure 1 +``` + +Notice that it says "Could not find module." This means that the package +containing the module in question is not available. To tell stack to use text, +you need to add it to your .cabal file — specifically in your build-depends +section, like this: + +``` +library + hs-source-dirs: src + exposed-modules: Lib + build-depends: base >= 4.7 && < 5 + -- This next line is the new one + , text + default-language: Haskell2010 +``` + +Now if we rerun `stack build`, we should get a successful result: + +``` +michael@d30748af6d3d:~/helloworld$ stack build +text-1.2.1.3: download +text-1.2.1.3: configure +text-1.2.1.3: build +text-1.2.1.3: install +helloworld-0.1.0.0: configure +Configuring helloworld-0.1.0.0... +helloworld-0.1.0.0: build +Preprocessing library helloworld-0.1.0.0... +[1 of 1] Compiling Lib ( src/Lib.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/Lib.o ) +In-place registering helloworld-0.1.0.0... +Preprocessing executable 'helloworld-exe' for helloworld-0.1.0.0... +[1 of 1] Compiling Main ( app/Main.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/helloworld-exe/helloworld-exe-tmp/Main.o ) [Lib changed] +Linking .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/helloworld-exe/helloworld-exe ... +helloworld-0.1.0.0: install +Installing library in +/home/michael/helloworld/.stack-work/install/x86_64-linux/lts-3.2/7.10.2/lib/x86_64-linux-ghc-7.10.2/helloworld-0.1.0.0-HI1deOtDlWiAIDtsSJiOtw +Installing executable(s) in +/home/michael/helloworld/.stack-work/install/x86_64-linux/lts-3.2/7.10.2/bin +Registering helloworld-0.1.0.0... +Completed all 2 actions. +``` + +This output means that the text package was downloaded, configured, built, and +locally installed. Once that was done, we moved on to building our local package +(helloworld). At no point did we need to ask stack to build dependencies — it +does so automatically. + +### extra-deps + +Let's try a more off-the-beaten-track package: the joke +[acme-missiles](http://www.stackage.org/package/acme-missiles) package. Our +source code is simple: + +```haskell +module Lib + ( someFunc + ) where + +import Acme.Missiles + +someFunc :: IO () +someFunc = launchMissiles +``` + +Again, we add this new dependency to the .cabal file like this: + +``` +library + hs-source-dirs: src + exposed-modules: Lib + build-depends: base >= 4.7 && < 5 + , text + -- This next line is the new one + , acme-missiles + default-language: Haskell2010 +``` + +However, rerunning `stack build` shows us the following error message: + +``` +michael@d30748af6d3d:~/helloworld$ stack build +While constructing the BuildPlan the following exceptions were encountered: + +-- While attempting to add dependency, + Could not find package acme-missiles in known packages + +-- Failure when adding dependencies: + acme-missiles: needed (-any), stack configuration has no specified version (latest applicable is 0.3) + needed for package: helloworld-0.1.0.0 + +Recommended action: try adding the following to your extra-deps in /home/michael/helloworld/stack.yaml +- acme-missiles-0.3 + +You may also want to try the 'stack solver' command +``` + +It says acme-missiles is "not present in build plan." This brings us to the next +major topic in using stack. + +## Curated package sets + +Remember above when `stack new` selected the lts-3.2 resolver for us? That +defined our build plan and available packages. When we tried using the +text package, it just worked, because it was part of the lts-3.2 *package set*. +But acme-missiles is not part of that package set, so building failed. + +To add this new dependency, we'll use the `extra-deps` field in stack.yaml to +define extra dependencies not present in the resolver. With that change, our +stack.yaml looks like: + +```yaml +flags: {} +packages: +- '.' +extra-deps: +- acme-missiles-0.3 # not in lts-3.2 +resolver: lts-3.2 +``` + +Now `stack build` will succeed. + +With that out of the way, let's dig a little bit more into these package sets, +also known as *snapshots*. We mentioned lts-3.2, and you can get quite a bit of +information about it at +[https://www.stackage.org/lts-3.2](https://www.stackage.org/lts-3.2), including: + +* The appropriate resolver value (`resolver: lts-3.2`, as we used above) +* The GHC version used +* A full list of all packages available in this snapshot +* The ability to perform a Hoogle search on the packages in this snapshot +* A [list of all modules](https://www.stackage.org/lts-3.2/docs) in a snapshot, + which an be useful when trying to determine which package to add to your + .cabal file + +You can also see a [list of all available +snapshots](https://www.stackage.org/snapshots). You'll notice two flavors: LTS +(for "Long Term Support") and Nightly. You can read more about them on the +[LTS Haskell Github page](https://github.com/fpco/lts-haskell#readme). If you're +not sure which to use, start with LTS Haskell (which stack will lean towards by +default as well). + +## Resolvers and changing your compiler version + +Let's explore package sets a bit further. Instead of lts-3.2, let's change our +stack.yaml file to use +[nightly-2015-08-26](https://www.stackage.org/nightly-2015-08-26). Rerunning +`stack build` will produce: + +``` +michael@d30748af6d3d:~/helloworld$ stack build +Downloaded nightly-2015-08-26 build plan. +Caching build plan +stm-2.4.4: configure +stm-2.4.4: build +stm-2.4.4: install +acme-missiles-0.3: configure +acme-missiles-0.3: build +acme-missiles-0.3: install +helloworld-0.1.0.0: configure +Configuring helloworld-0.1.0.0... +helloworld-0.1.0.0: build +Preprocessing library helloworld-0.1.0.0... +In-place registering helloworld-0.1.0.0... +Preprocessing executable 'helloworld-exe' for helloworld-0.1.0.0... +Linking .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/helloworld-exe/helloworld-exe ... +helloworld-0.1.0.0: install +Installing library in +/home/michael/helloworld/.stack-work/install/x86_64-linux/nightly-2015-08-26/7.10.2/lib/x86_64-linux-ghc-7.10.2/helloworld-0.1.0.0-6cKaFKQBPsi7wB4XdqRv8w +Installing executable(s) in +/home/michael/helloworld/.stack-work/install/x86_64-linux/nightly-2015-08-26/7.10.2/bin +Registering helloworld-0.1.0.0... +Completed all 3 actions. +``` + +We can also change resolvers on the command line, which can be useful in a +Continuous Integration (CI) setting, like on Travis. For example: + +``` +michael@d30748af6d3d:~/helloworld$ stack --resolver lts-3.1 build +Downloaded lts-3.1 build plan. +Caching build plan +stm-2.4.4: configure +# Rest is the same, no point copying it +``` + +When passed on the command line, you also get some additional "short-cut" +versions of resolvers: `--resolver nightly` will use the newest Nightly resolver +available, `--resolver lts` will use the newest LTS, and `--resolver lts-2` will +use the newest LTS in the 2.X series. The reason these are only available on the +command line and not in your stack.yaml file is that using them: + +1. Will slow down your build (since stack then needs to download information on + the latest available LTS each time it builds) +2. Produces unreliable results (since a build run today may proceed differently + tomorrow because of changes outside of your control) + +### Changing GHC versions + +Finally, let's try using an older LTS snapshot. We'll use the newest 2.X +snapshot: + +``` +michael@d30748af6d3d:~/helloworld$ stack --resolver lts-2 build +Selected resolver: lts-2.22 +Downloaded lts-2.22 build plan. +Caching build plan +No GHC found, expected version 7.8.4 (x86_64) (based on resolver setting in /home/michael/helloworld/stack.yaml). Try running stack setup +``` + +This fails, because GHC 7.8.4 (which lts-2.22 uses) is not available on our +system. So, we see that different LTS versions (2 vs 3 in this case) use +different GHC versions. Now, how do we get the right GHC version after changing +the LTS version? One answer is to use `stack setup` like we did above, this +time with the `--resolver lts-2` option. However, there's another method worth +mentioning: the `--install-ghc` flag. + +``` +michael@d30748af6d3d:~/helloworld$ stack --resolver lts-2 --install-ghc build +Selected resolver: lts-2.22 +Downloaded ghc-7.8.4. +Installed GHC. +stm-2.4.4: configure +# Mostly same as before, nothing interesting to see +``` + +What's nice about `--install-ghc` is: + +1. You don't need to have an extra step in your build script +2. It only requires downloading the information on latest snapshots once + +As mentioned above, the default behavior of stack is to *not* install new +versions of GHC automatically. We want to avoid surprising users with large +downloads/installs. The `--install-ghc` flag simply changes that default +behavior. + +### Other resolver values + +We've mentioned `nightly-YYYY-MM-DD` and `lts-X.Y` values for the resolver. +There are actually other options available, and the list will grow over time. +At the time of writing: + +* `ghc-X.Y.Z`, for requiring a specific GHC version but no additional packages +* Experimental GHCJS support +* Experimental custom snapshot support + +The most up-to-date information can always be found in the +[stack.yaml documentation](yaml_configuration.md#resolver). + +## Existing projects + +Alright, enough playing around with simple projects. Let's take an open source +package and try to build it. We'll be ambitious and use +[yackage](https://www.stackage.org/package/yackage), a local package server +using [Yesod](http://www.yesodweb.com/). To get the code, we'll use the `stack +unpack` command: + +``` +cueball:~$ stack unpack yackage-0.8.0 +Unpacked yackage-0.8.0 to /var/home/harendra/yackage-0.8.0/ +cueball:~$ cd yackage-0.8.0/ +``` + +### stack init +This new directory does not have a stack.yaml file, so we need to make one +first. We could do it by hand, but let's be lazy instead with the `stack init` +command: + +``` +cueball:~/yackage-0.8.0$ stack init +Using cabal packages: +- yackage.cabal + +Selecting the best among 6 snapshots... + +* Matches lts-4.1 + +Selected resolver: lts-4.1 +Initialising configuration using resolver: lts-4.1 +Total number of user packages considered: 1 +Writing configuration to file: stack.yaml +All done. +``` + +stack init does quite a few things for you behind the scenes: + +* Finds all of the .cabal files in your current directory and subdirectories + (unless you use `--ignore-subdirs`) and determines the packages and versions + they require +* Finds the best combination of snapshot and package flags that allows everything to + compile with minimum external dependencies +* It tries to look for the best matching snapshot from latest LTS, latest + nightly, other LTS versions in that order + +Assuming it finds a match, it will write your stack.yaml file, and everything +will work. + +#### External Dependencies + +Given that LTS Haskell and Stackage Nightly have ~1400 of the most common +Haskell packages, this will often be enough to build most packages. However, +at times, you may find that not all dependencies required may be available in +the Stackage snapshots. + +Let's simulate an unsatisfied dependency by adding acme-missiles to our +build-depends and re-initing: + +``` +cueball:~/yackage-0.8.0$ stack init --force +Using cabal packages: +- yackage.cabal + +Selecting the best among 6 snapshots... + +* Partially matches lts-4.1 + acme-missiles not found + - yackage requires -any + - yackage flags: upload = True + +* Partially matches nightly-2016-01-16 + acme-missiles not found + - yackage requires -any + - yackage flags: upload = True + +* Partially matches lts-3.22 + acme-missiles not found + - yackage requires -any + - yackage flags: upload = True + +. +. +. + +Selected resolver: lts-4.1 +Resolver 'lts-4.1' does not have all the packages to match your requirements. + acme-missiles not found + - yackage requires -any + - yackage flags: upload = True + +However, you can try '--solver' to use external packages. +``` + +stack has tested six different snapshots, and in every case discovered that +acme-missiles is not available. In the end it suggested that you use the +`--solver` command line switch if you want to use packages outside Stackage. So +let's give it a try: + + +``` +cueball:~/yackage-0.8.0$ stack init --force --solver +Using cabal packages: +- yackage.cabal + +Selecting the best among 6 snapshots... + +* Partially matches lts-4.1 + acme-missiles not found + - yackage requires -any + - yackage flags: upload = True + +. +. +. + +Selected resolver: lts-4.1 +*** Resolver lts-4.1 will need external packages: + acme-missiles not found + - yackage requires -any + - yackage flags: upload = True + +Using resolver: lts-4.1 +Using compiler: ghc-7.10.3 +Asking cabal to calculate a build plan... +Trying with packages from lts-4.1 as hard constraints... +Successfully determined a build plan with 3 external dependencies. +Initialising configuration using resolver: lts-4.1 +Total number of user packages considered: 1 +Warning! 3 external dependencies were added. +Overwriting existing configuration file: stack.yaml +All done. +``` + +As you can verify by viewing stack.yaml, three external dependencies were added +by stack init: + +``` +# Packages to be pulled from upstream that are not in the resolver (e.g., acme-missiles-0.3) +extra-deps: +- acme-missiles-0.3 +- text-1.2.2.0 +- yaml-0.8.15.2 +``` + +Of course, you could have added the external dependencies by manually editing +stack.yaml but stack init does the hard work for you. + +#### Excluded Packages + +Sometimes multiple packages in your project may have conflicting requirements. +In that case `stack init` will fail, so what do you do? + +You could manually create stack.yaml by omitting some packages to resolve the +conflict. Alternatively you can ask `stack init` to do that for you by +specifying `--omit-packages` flag on the command line. Let's see how that +works. + +To simulate a conflict we will use acme-missiles-0.3 in yackage and we will +also copy yackage.cabal to another directory and change the name of the file +and package to yackage-test. In this new package we will use acme-missiles-0.2 +instead. Let's see what happens when we run solver: + +``` +cueball:~/yackage-0.8.0$ stack init --force --solver --omit-packages +Using cabal packages: +- yackage.cabal +- example/yackage-test.cabal + +Selecting the best among 6 snapshots... + +* Partially matches lts-4.2 + acme-missiles not found + - yackage requires ==0.3 + - yackage-test requires ==0.2 + - yackage flags: upload = True + - yackage-test flags: upload = True +. +. +. + +*** Failed to arrive at a workable build plan. +*** Ignoring package: yackage-test +*** Resolver lts-4.2 will need external packages: + acme-missiles not found + - yackage requires ==0.3 + - yackage flags: upload = True + +Using resolver: lts-4.2 +Using compiler: ghc-7.10.3 +Asking cabal to calculate a build plan... +Trying with packages from lts-4.2 as hard constraints... +Successfully determined a build plan with 3 external dependencies. +Initialising configuration using resolver: lts-4.2 +Total number of user packages considered: 2 +Warning! Ignoring 1 packages due to dependency conflicts: + - "example/yackage-test.cabal" + +Warning! 3 external dependencies were added. +Overwriting existing configuration file: stack.yaml +All done. +``` + +Looking at `stack.yaml`, you will see that the excluded packages have been +commented out: + +``` +# Local packages, usually specified by relative directory name +packages: +- '.' +# The following packages have been ignored due to incompatibility with the resolver compiler or dependency conflicts with other packages +#- example/ +``` + +In case wrong packages are excluded you can uncomment the right one and comment +the other one. + +Packages may get excluded due to conflicting requirements among user packages +or due to conflicting requirements between a user package and the resolver +compiler. If all of the packages have a conflict with the compiler then all of +them may get commented out. + +When packages are commented out you will see a warning every time you run a +command which needs the config file. The warning can be disabled by editing the +config file and removing it. + +#### Using a specific resolver + +Sometimes you may want to use a specific resolver for your project instead of +`stack init` picking one for you. You can do that by using `stack init +--resolver `. + +You can also init with a compiler resolver if you do not want to use a +snapshot. That will result in all of your project's dependencies being put +under the `extra-deps` section. + +#### Installing the compiler + +You can install the required compiler if not already installed by using the +`--install-ghc` flag with the `stack init` command. + +#### Miscellaneous and diagnostics + +_Add selected packages_: If you want to use only selected packages from your +project directory you can do so by explicitly specifying the package directories +on the command line. + +_Duplicate package names_: If multiple packages under the directory tree have +same name, stack init will report those and automatically ignore one of them. + +_Ignore subdirectories_: By default stack init searches all the subdirectories +for .cabal files. If you do not want that then you can use `--ignore-subdirs` +command line switch. + +_Cabal warnings_: stack init will show warnings if there were issues in reading +a cabal package file. You may want to pay attention to the warnings as +sometimes they may result in incomprehensible errors later on during dependency +solving. + +_Package naming_: If the `Name` field defined in a cabal file does not match +with the cabal file name then `stack init` will refuse to continue. + +_Cabal install errors_: stack init uses `cabal-install` to determine external +dependencies. When cabal-install encounters errors, cabal errors are displayed +as is by stack init for diagnostics. + +_User warnings_: When packages are excluded or external dependencies added +stack will show warnings every time configuration file is loaded. You can +suppress the warnings by editing the config file and removing the warnings from +it. You may see something like this: + +``` +cueball:~/yackage-0.8.0$ stack build +Warning: Some packages were found to be incompatible with the resolver and have been left commented out in the packages section. +Warning: Specified resolver could not satisfy all dependencies. Some external packages have been added as dependencies. +You can suppress this message by removing it from stack.yaml + +``` +### stack solver + +While `stack init` is used to create stack configuration file from existing +cabal files, `stack solver` can be used to fine tune or fix an existing stack +configuration file. + +`stack solver` uses the existing file as a constraint. For example it will +use only those packages specified in the existing config file or use existing +external dependencies as constraints to figure out other dependencies. + +Let's try `stack solver` to verify the config that we generated earlier with +`stack init`: + +``` +cueball:~/yackage-0.8.0$ stack solver +Using configuration file: stack.yaml +The following packages are missing from the config: +- example/yackage-test.cabal + +Using cabal packages: +- yackage.cabal + +Using resolver: lts-4.2 +Using compiler: ghc-7.10.3 +Asking cabal to calculate a build plan... +Trying with packages from lts-4.2 and 3 external packages as hard constraints... +Successfully determined a build plan with 3 external dependencies. +No changes needed to stack.yaml +``` + +It says there are no changes needed to your config. Notice that it also reports +`example/yackage-test.cabal` as missing from the config. It was purposely +omitted by `stack init` to resolve a conflict. + +Sometimes `stack init` may not be able to give you a perfect configuration. In +that case, you can tweak the configuration file as per your requirements and then +run `stack solver`, it will check the file and suggest or apply any fixes +needed. + +For example, if `stack init` ignored certain packages due to name conflicts or +dependency conflicts, the choice that `stack init` made may not be the correct +one. In that case you can revert the choice and use solver to fix things. + +Let's try commenting out `.` and uncommenting `examples/` in our previously +generated `stack.yaml` and then run `stack solver`: + +``` +cueball:~/yackage-0.8.0$ stack solver + +Using configuration file: stack.yaml +The following packages are missing from the config: +- yackage.cabal + +Using cabal packages: +- example/yackage-test.cabal + +. +. +. + +Retrying with packages from lts-4.2 and 3 external packages as preferences... +Successfully determined a build plan with 5 external dependencies. + +The following changes will be made to stack.yaml: +* Resolver is lts-4.2 +* Dependencies to be added + extra-deps: + - acme-missiles-0.2 + - email-validate-2.2.0 + - tar-0.5.0.1 + +* Dependencies to be deleted + extra-deps: + - acme-missiles-0.3 + +To automatically update stack.yaml, rerun with '--update-config' +``` + +Due to the change that we made, solver suggested some new dependencies. +By default it does not make changes to the config. As it suggested you can use +`--update-config` to make changes to the config. + +NOTE: You should probably back up your stack.yaml before doing this, such as +committing to Git/Mercurial/Darcs. + +Sometimes, you may want to use specific versions of certain packages for your +project. To do that you can fix those versions by specifying them in the +extra-deps section and then use `stack solver` to figure out whether it is +feasible to use those or what other dependencies are needed as a result. + +If you want to change the resolver for your project, you can run `stack solver +--resolver ` and it will figure out the changes needed for you. + +Let's see what happens if we change the resolver to lts-2.22: + +``` +cueball:~/yackage-0.8.0$ stack solver --resolver lts-2.22 +Using configuration file: stack.yaml +The following packages are missing from the config: +- yackage.cabal + +Using cabal packages: +- example/yackage-test.cabal + +Using resolver: lts-2.22 +Using compiler: ghc-7.8.4 + +. +. +. + +Retrying with packages from lts-2.22 and 3 external packages as preferences... +Successfully determined a build plan with 19 external dependencies. + +The following changes will be made to stack.yaml: +* Resolver is lts-2.22 +* Flags to be added + flags: + - old-locale: true + +* Dependencies to be added + extra-deps: + - acme-missiles-0.2 + - aeson-0.10.0.0 + - aeson-compat-0.3.0.0 + - attoparsec-0.13.0.1 + - conduit-extra-1.1.9.2 + - email-validate-2.2.0 + - hex-0.1.2 + - http-api-data-0.2.2 + - http2-1.1.0 + - persistent-2.2.4 + - persistent-template-2.1.5 + - primitive-0.6.1.0 + - tar-0.5.0.1 + - unix-time-0.3.6 + - vector-0.11.0.0 + - wai-extra-3.0.14 + - warp-3.1.3.1 + +* Dependencies to be deleted + extra-deps: + - acme-missiles-0.3 + +To automatically update stack.yaml, rerun with '--update-config' +``` + +As you can see, it automatically suggested changes in `extra-deps` due to the +change of resolver. + +## Different databases + +Time to take a short break from hands-on examples and discuss a little +architecture. stack has the concept of multiple *databases*. A database +consists of a GHC package database (which contains the compiled version of a +library), executables, and a few other things as well. To give you an idea: + +``` +michael@d30748af6d3d:~/helloworld$ ls .stack-work/install/x86_64-linux/lts-3.2/7.10.2/ +bin doc flag-cache lib pkgdb +``` + +Databases in stack are *layered*. For example, the database listing we just gave +is called a *local* database. That is layered on top of a *snapshot* database, +which contains the libraries and executables specified in the snapshot itself. +Finally, GHC itself ships with a number of libraries and executables, which +forms the *global* database. To get a quick idea of this, we can look at the +output of the `stack exec ghc-pkg list` command in our helloworld project: + +``` +/home/michael/.stack/programs/x86_64-linux/ghc-7.10.2/lib/ghc-7.10.2/package.conf.d + Cabal-1.22.4.0 + array-0.5.1.0 + base-4.8.1.0 + bin-package-db-0.0.0.0 + binary-0.7.5.0 + bytestring-0.10.6.0 + containers-0.5.6.2 + deepseq-1.4.1.1 + directory-1.2.2.0 + filepath-1.4.0.0 + ghc-7.10.2 + ghc-prim-0.4.0.0 + haskeline-0.7.2.1 + hoopl-3.10.0.2 + hpc-0.6.0.2 + integer-gmp-1.0.0.0 + pretty-1.1.2.0 + process-1.2.3.0 + rts-1.0 + template-haskell-2.10.0.0 + terminfo-0.4.0.1 + time-1.5.0.1 + transformers-0.4.2.0 + unix-2.7.1.0 + xhtml-3000.2.1 +/home/michael/.stack/snapshots/x86_64-linux/nightly-2015-08-26/7.10.2/pkgdb + stm-2.4.4 +/home/michael/helloworld/.stack-work/install/x86_64-linux/nightly-2015-08-26/7.10.2/pkgdb + acme-missiles-0.3 + helloworld-0.1.0.0 +``` + +Notice that acme-missiles ends up in the *local* database. Anything which is +not installed from a snapshot ends up in the local database. This includes: +your own code, extra-deps, and in some cases even snapshot packages, if you +modify them in some way. The reason we have this structure is that: + +* it lets multiple projects reuse the same binary builds of many snapshot + packages, +* but doesn't allow different projects to "contaminate" each other by putting + non-standard content into the shared snapshot database + +Typically, the process by which a snapshot package is marked as modified is +referred to as "promoting to an extra-dep," meaning we treat it just like a +package in the extra-deps section. This happens for a variety of reasons, +including: + +* changing the version of the snapshot package +* changing build flags +* one of the packages that the package depends on has been promoted to an + extra-dep + +As you probably guessed, there are multiple snapshot databases available, e.g.: + +``` +michael@d30748af6d3d:~/helloworld$ ls ~/.stack/snapshots/x86_64-linux/ +lts-2.22 lts-3.1 lts-3.2 nightly-2015-08-26 +``` + +These databases don't get layered on top of each other; they are each used +separately. + +In reality, you'll rarely — if ever — interact directly with these databases, +but it's good to have a basic understanding of how they work so you can +understand why rebuilding may occur at different points. + +## The build synonyms + +Let's look at a subset of the `stack --help` output: + +``` +build Build the package(s) in this directory/configuration +install Shortcut for 'build --copy-bins' +test Shortcut for 'build --test' +bench Shortcut for 'build --bench' +haddock Shortcut for 'build --haddock' +``` + +Note that four of these commands are just synonyms for the `build` command. They +are provided for convenience for common cases (e.g., `stack test` instead of +`stack build --test`) and so that commonly expected commands just work. + +What's so special about these commands being synonyms? It allows us to make +much more composable command lines. For example, we can have a command that +builds executables, generates Haddock documentation (Haskell API-level docs), +and builds and runs your test suites, with: + +``` +stack build --haddock --test +``` + +You can even get more inventive as you learn about other flags. For example, +take the following: + +``` +stack build --pedantic --haddock --test --exec "echo Yay, it succeeded" --file-watch +``` + +This will: + +* turn on all warnings and errors +* build your library and executables +* generate Haddocks +* build and run your test suite +* run the command `echo Yay, it succeeded` when that completes +* after building, watch for changes in the files used to build the project, and + kick off a new build when done + +### install and copy-bins + +It's worth calling out the behavior of the install command and `--copy-bins` +option, since this has confused a number of users (especially when compared to +behavior of other tools like cabal-install). The `install` command does +precisely one thing in addition to the build command: it copies any generated +executables to the local bin path. You may recognize the default value for that +path: + +``` +michael@d30748af6d3d:~/helloworld$ stack path --local-bin-path +/home/michael/.local/bin +``` + +That's why the download page recommends adding that directory to your `PATH` +environment variable. This feature is convenient, because now you can simply +run `executable-name` in your shell instead of having to run `stack exec +executable-name` from inside your project directory. + +Since it's such a point of confusion, let me list a number of things stack does +*not* do specially for the install command: + +* stack will always build any necessary dependencies for your code. The install + command is not necessary to trigger this behavior. If you just want to build a + project, run `stack build`. +* stack will *not* track which files it's copied to your local bin path nor + provide a way to automatically delete them. There are many great tools out + there for managing installation of binaries, and stack does not attempt to + replace those. +* stack will not necessarily be creating a relocatable executable. If your + executables hard-codes paths, copying the executable will not change those + hard-coded paths. + * At the time of writing, there's no way to change those kinds of paths with + stack, but see [issue #848 about + --prefix](https://github.com/commercialhaskell/stack/issues/848) for + future plans. + +That's really all there is to the install command: for the simplicity of what +it does, it occupies a much larger mental space than is warranted. + +## Targets, locals, and extra-deps + +We haven't discussed this too much yet, but, in addition to having a number of +synonyms *and* taking a number of options on the command line, the build command +*also* takes many arguments. These are parsed in different ways, and can be used +to achieve a high level of flexibility in telling stack exactly what you want +to build. + +We're not going to cover the full generality of these arguments here; instead, +there's [documentation covering the full build command +syntax](build_command.md). +Here, we'll just point out a few different types of arguments: + +* You can specify a *package name*, e.g. `stack build vector`. + * This will attempt to build the vector package, whether it's a local + package, in your extra-deps, in your snapshot, or just available upstream. + If it's just available upstream but not included in your locals, + extra-deps, or snapshot, the newest version is automatically promoted to + an extra-dep. +* You can also give a *package identifier*, which is a package name plus + version, e.g. `stack build yesod-bin-1.4.14`. + * This is almost identical to specifying a package name, except it will (1) + choose the given version instead of latest, and (2) error out if the given + version conflicts with the version of a local package. +* The most flexibility comes from specifying individual *components*, e.g. + `stack build helloworld:test:helloworld-test` says "build the test suite + component named helloworld-test from the helloworld package." + * In addition to this long form, you can also shorten it by skipping what + type of component it is, e.g. `stack build helloworld:helloworld-test`, or + even skip the package name entirely, e.g. `stack build :helloworld-test`. +* Finally, you can specify individual *directories* to build to trigger building + of any local packages included in those directories or subdirectories. + +When you give no specific arguments on the command line (e.g., `stack build`), +it's the same as specifying the names of all of your local packages. If you +just want to build the package for the directory you're currently in, you can +use `stack build .`. + +### Components, --test, and --bench + +Here's one final important yet subtle point. Consider our helloworld package: +it has a library component, an executable helloworld-exe, and a test suite +helloworld-test. When you run `stack build helloworld`, how does it know which +ones to build? By default, it will build the library (if any) and all of the +executables but ignore the test suites and benchmarks. + +This is where the `--test` and `--bench` flags come into play. If you use them, +those components will also be included. So `stack build --test helloworld` will +end up including the helloworld-test component as well. + +You can bypass this implicit adding of components by being much more explicit, +and stating the components directly. For example, the following will not build +the helloworld-exe executable: + +``` +michael@d30748af6d3d:~/helloworld$ stack clean +michael@d30748af6d3d:~/helloworld$ stack build :helloworld-test +helloworld-0.1.0.0: configure (test) +Configuring helloworld-0.1.0.0... +helloworld-0.1.0.0: build (test) +Preprocessing library helloworld-0.1.0.0... +[1 of 1] Compiling Lib ( src/Lib.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/Lib.o ) +In-place registering helloworld-0.1.0.0... +Preprocessing test suite 'helloworld-test' for helloworld-0.1.0.0... +[1 of 1] Compiling Main ( test/Spec.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/helloworld-test/helloworld-test-tmp/Main.o ) +Linking .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/helloworld-test/helloworld-test ... +helloworld-0.1.0.0: test (suite: helloworld-test) +Test suite not yet implemented +``` + +We first cleaned our project to clear old results so we know exactly what stack +is trying to do. Notice that it builds the helloworld-test test suite, and the +helloworld library (since it's used by the test suite), but it does not build +the helloworld-exe executable. + +And now the final point: the last line shows that our command also *runs* the +test suite it just built. This may surprise some people who would expect tests +to only be run when using `stack test`, but this design decision is what allows +the `stack build` command to be as composable as it is (as described +previously). The same rule applies to benchmarks. To spell it out completely: + +* The --test and --bench flags simply state which components of a package should + be built, if no explicit set of components is given +* The default behavior for any test suite or benchmark component which has been + built is to also run it + +You can use the `--no-run-tests` and `--no-run-benchmarks` (from stack-0.1.4.0 +and on) flags to disable running of these components. You can also use +`--no-rerun-tests` to prevent running a test suite which has already passed and +has not changed. + +NOTE: stack doesn't build or run test suites and benchmarks for non-local +packages. This is done so that running a command like `stack test` doesn't need +to run 200 test suites! + +## Multi-package projects + +Until now, everything we've done with stack has used a single-package project. +However, stack's power truly shines when you're working on multi-package +projects. All the functionality you'd expect to work just does: dependencies +between packages are detected and respected, dependencies of all packages are +just as one cohesive whole, and if anything fails to build, the build commands +exits appropriately. + +Let's demonstrate this with the wai-app-static and yackage packages: + +``` +michael@d30748af6d3d:~$ mkdir multi +michael@d30748af6d3d:~$ cd multi/ +michael@d30748af6d3d:~/multi$ stack unpack wai-app-static-3.1.1 yackage-0.8.0 +wai-app-static-3.1.1: download +Unpacked wai-app-static-3.1.1 to /home/michael/multi/wai-app-static-3.1.1/ +Unpacked yackage-0.8.0 to /home/michael/multi/yackage-0.8.0/ +michael@d30748af6d3d:~/multi$ stack init +Writing default config file to: /home/michael/multi/stack.yaml +Basing on cabal files: +- /home/michael/multi/yackage-0.8.0/yackage.cabal +- /home/michael/multi/wai-app-static-3.1.1/wai-app-static.cabal + +Checking against build plan lts-3.2 +Selected resolver: lts-3.2 +Wrote project config to: /home/michael/multi/stack.yaml +michael@d30748af6d3d:~/multi$ stack build --haddock --test +# Goes off to build a whole bunch of packages +``` + +If you look at the stack.yaml, you'll see exactly what you'd expect: + +```yaml +flags: + yackage: + upload: true + wai-app-static: + print: false +packages: +- yackage-0.8.0/ +- wai-app-static-3.1.1/ +extra-deps: [] +resolver: lts-3.2 +``` + +Notice that multiple directories are listed in the `packages` key. + +In addition to local directories, you can also refer to packages available in a +Git repository or in a tarball over HTTP/HTTPS. This can be useful for using a +modified version of a dependency that hasn't yet been released upstream. + +Please note that when adding upstream packages directly to your project it is +important to distinguish _local packages_ from the upstream _dependency +packages_. Otherwise you may have trouble running `stack GHCi`. See +[stack.yaml documentation](yaml_configuration.md#packages) for more details. + +## Flags and GHC options + +There are two common ways to alter how a package will install: with Cabal flags +and with GHC options. + +### Cabal flag management + +In the stack.yaml file above, you can see that `stack init` has detected that — +for the yackage package — the upload flag can be set to true, and for +wai-app-static, the print flag to false (it's chosen those values because +they're the default flag values, and their dependencies are compatible with the +snapshot we're using.) To change a flag setting, we can use the command +line `--flag` option: + + stack build --flag yackage:-upload + +This means: when compiling the yackage package, turn off the upload flag (thus +the `-`). Unlike other tools, stack is explicit about which package's flag you +want to change. It does this for two reasons: + +1. There's no global meaning for Cabal flags, and therefore two packages can + use the same flag name for completely different things. +2. By following this approach, we can avoid unnecessarily recompiling snapshot + packages that happen to use a flag that we're using. + +You can also change flag values on the command line for extra-dep and snapshot +packages. If you do this, that package will automatically be promoted to an +extra-dep, since the build plan is different than what the plan snapshot +definition would entail. + +### GHC options + +GHC options follow a similar logic as in managing Cabal flags, with a few +nuances to adjust for common use cases. Let's consider: + + stack build --ghc-options="-Wall -Werror" + +This will set the `-Wall -Werror` options for all *local targets*. Note that +this will not affect extra-dep and snapshot packages at all. This design +provides us with reproducible and fast builds. + +(By the way: the above GHC options have a special convenience flag: +`--pedantic`.) + +There's one extra nuance about command line GHC options: Since they only apply +to local targets, if you change your local targets, they will no longer apply +to other packages. Let's play around with an example from the wai repository, +which includes the wai and warp packages, the latter depending on the former. +If we run: + + stack build --ghc-options=-O0 wai + +It will build all of the dependencies of wai, and then build wai with all +optimizations disabled. Now let's add in warp as well: + + stack build --ghc-options=-O0 wai warp + +This builds the additional dependencies for warp, and then builds warp with +optimizations disabled. Importantly: it does not rebuild wai, since wai's +configuration has not been altered. Now the surprising case: + +``` +michael@d30748af6d3d:~/wai$ stack build --ghc-options=-O0 warp +wai-3.0.3.0-5a49351d03cba6cbaf906972d788e65d: unregistering (flags changed from ["--ghc-options","-O0"] to []) +warp-3.1.3-a91c7c3108f63376877cb3cd5dbe8a7a: unregistering (missing dependencies: wai) +wai-3.0.3.0: configure +``` + +You may expect this to be a no-op: neither wai nor warp has changed. However, +stack will instead recompile wai with optimizations enabled again, and then +rebuild warp (with optimizations disabled) against this newly built wai. The +reason: reproducible builds. If we'd never built wai or warp before, trying to +build warp would necessitate building all of its dependencies, and it would do +so with default GHC options (optimizations enabled). This dependency would +include wai. So when we run: + + stack build --ghc-options=-O0 warp + +We want its behavior to be unaffected by any previous build steps we took. +While this specific corner case does catch people by surprise, the overall goal +of reproducible builds is- in the stack maintainers' views- worth the +confusion. + +Final point: if you have GHC options that you'll be regularly passing to your +packages, you can add them to your stack.yaml file (starting with +stack-0.1.4.0). See [the documentation section on +ghc-options](yaml_configuration.md#ghc-options) +for more information. + +## path + +NOTE: That's it, the heavy content of this guide is done! Everything from here +on out is simple explanations of commands. Congratulations! + +Generally, you don't need to worry about where stack stores various files. But +some people like to know this stuff. That's when the `stack path` command is +useful. + +``` +michael@d30748af6d3d:~/wai$ stack path +global-stack-root: /home/michael/.stack +stack-root: /home/michael/.stack +project-root: /home/michael/wai +config-location: /home/michael/wai/stack.yaml +bin-path: /home/michael/.stack/snapshots/x86_64-linux/lts-2.17/7.8.4/bin:/home/michael/.stack/programs/x86_64-linux/ghc-7.8.4/bin:/home/michael/.stack/programs/x86_64-linux/ghc-7.10.2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin +programs: /home/michael/.stack/programs/x86_64-linux +compiler: /home/michael/.stack/programs/x86_64-linux/ghc-7.8.4/bin/ghc +compiler-bin: /home/michael/.stack/programs/x86_64-linux/ghc-7.8.4/bin +local-bin-path: /home/michael/.local/bin +extra-include-dirs: +extra-library-dirs: +snapshot-pkg-db: /home/michael/.stack/snapshots/x86_64-linux/lts-2.17/7.8.4/pkgdb +local-pkg-db: /home/michael/wai/.stack-work/install/x86_64-linux/lts-2.17/7.8.4/pkgdb +global-pkg-db: /home/michael/.stack/programs/x86_64-linux/ghc-7.8.4/lib/ghc-7.8.4/package.conf.d +ghc-package-path: /home/michael/wai/.stack-work/install/x86_64-linux/lts-2.17/7.8.4/pkgdb:/home/michael/.stack/snapshots/x86_64-linux/lts-2.17/7.8.4/pkgdb:/home/michael/.stack/programs/x86_64-linux/ghc-7.8.4/lib/ghc-7.8.4/package.conf.d +snapshot-install-root: /home/michael/.stack/snapshots/x86_64-linux/lts-2.17/7.8.4 +local-install-root: /home/michael/wai/.stack-work/install/x86_64-linux/lts-2.17/7.8.4 +snapshot-doc-root: /home/michael/.stack/snapshots/x86_64-linux/lts-2.17/7.8.4/doc +local-doc-root: /home/michael/wai/.stack-work/install/x86_64-linux/lts-2.17/7.8.4/doc +dist-dir: .stack-work/dist/x86_64-linux/Cabal-1.18.1.5 +local-hpc-root: /home/michael/wai/.stack-work/install/x86_64-linux/lts-2.17/7.8.4/hpc +``` + +In addition, `stack path` accepts command line arguments to state which of +these keys you're interested in, which can be convenient for scripting. As a +simple example, let's find out which versions of GHC are installed locally: + +``` +michael@d30748af6d3d:~/wai$ ls $(stack path --programs)/*.installed +/home/michael/.stack/programs/x86_64-linux/ghc-7.10.2.installed +/home/michael/.stack/programs/x86_64-linux/ghc-7.8.4.installed +``` + +(Yes, that command requires a \*nix shell, and likely won't run on Windows.) + +While we're talking about paths, to wipe our stack install completely, here's +what needs to be removed: + +1. The stack executable itself +2. The stack root, e.g. `$HOME/.stack` on non-Windows systems. + * See `stack path --stack-root` + * On Windows, you will also need to delete `stack path --programs` +3. Any local `.stack-work` directories inside a project + +## exec + +We've already used `stack exec` used multiple times in this guide. As you've +likely already guessed, it allows you to run executables, but with a slightly +modified environment. In particular: `stack exec` looks for executables on +stack's bin paths, and sets a few additional environment variables (like +`GHC_PACKAGE_PATH`, which tells GHC which package databases to use). + +If you want to see exactly what the modified environment looks like, try: + + stack exec env + +The only issue is how to distinguish flags to be passed to stack versus those +for the underlying program. Thanks to the optparse-applicative library, stack +follows the Unix convention of `--` to separate these, e.g.: + +``` +michael@d30748af6d3d:~$ stack exec --package stm -- echo I installed the stm package via --package stm +Run from outside a project, using implicit global project config +Using latest snapshot resolver: lts-3.2 +Writing global (non-project-specific) config file to: /home/michael/.stack/global/stack.yaml +Note: You can change the snapshot via the resolver field there. +I installed the stm package via --package stm +``` + +Flags worth mentioning: + +* `--package foo` can be used to force a package to be installed before running + the given command. +* `--no-ghc-package-path` can be used to stop the `GHC_PACKAGE_PATH` environment + variable from being set. Some tools — notably cabal-install — do not behave + well with that variable set. + +## ghci (the repl) + +GHCi is the interactive GHC environment, a.k.a. the REPL. You *could* access it +with: + + stack exec ghci + +But that won't load up locally written modules for access. For that, use the +`stack ghci` command. To then load modules from your project, use the `:m` +command (for "module") followed by the module name. + +IMPORTANT NOTE: If you have added upstream packages to your project please make +sure to mark them as *dependency package*s for faster and reliable usage of +`stack gchi`. Otherwise GHCi may have trouble due to conflicts of compilation +flags or having to unnecessarily interpret too many modules. See +[stack.yaml documentation](yaml_configuration.md#packages) to learn how to mark +a package as a *dependency package*. + +## ghc/runghc + +You'll sometimes want to just compile (or run) a single Haskell source file, +instead of creating an entire Cabal package for it. You can use `stack exec +ghc` or `stack exec runghc` for that. As simple helpers, we also provide the +`stack ghc` and `stack runghc` commands, for these common cases. + +## script interpreter + +stack also offers a very useful feature for running files: a script +interpreter. For too long have Haskellers felt shackled to bash or Python +because it's just too hard to create reusable source-only Haskell scripts. +stack attempts to solve that. + +You can use `stack ` to execute a Haskell source file or specify +`stack` as the interpreter using a shebang line on a Unix like operating systems. +Additional stack options can be specified using a special Haskell comment in +the source file to specify dependencies and automatically install them before +running the file. + +An example will be easiest to understand: + +``` +michael@d30748af6d3d:~$ cat turtle.hs +#!/usr/bin/env stack +-- stack --resolver lts-3.2 --install-ghc runghc --package turtle +{-# LANGUAGE OverloadedStrings #-} +import Turtle +main = echo "Hello World!" +michael@d30748af6d3d:~$ chmod +x turtle.hs +michael@d30748af6d3d:~$ ./turtle.hs +Run from outside a project, using implicit global project config +Using resolver: lts-3.2 specified on command line +hashable-1.2.3.3: configure +# installs some more dependencies +Completed all 22 actions. +Hello World! +michael@d30748af6d3d:~$ ./turtle.hs +Run from outside a project, using implicit global project config +Using resolver: lts-3.2 specified on command line +Hello World! +``` + +The first run can take a while (as it has to download GHC if necessary and build +dependencies), but subsequent runs are able to reuse everything already built, +and are therefore quite fast. + +The first line in the source file is the usual "shebang" to use stack as a +script interpreter. The second line, is a Haskell comment providing additional +options to stack (due to the common limitation of the "shebang" line only being +allowed a single argument). In this case, the options tell stack to use the +lts-3.2 resolver, automatically install GHC if it is not already installed, and +ensure the turtle package is available. + +If you're on Windows: you can run `stack turtle.hs` instead of `./turtle.hs`. +The shebang line is not required in that case. + +### Specifying interpreter options + +The stack interpreter options comment must specify a single valid stack command +line, starting with `stack` as the command followed by the stack options to use +for executing this file. The comment must always be on the line immediately +following the shebang line when the shebang line is present otherwise it must +be the first line in the file. The comment must always start in the first +column of the line. + +When many options are needed a block style comment may be more convenient to +split the command on multiple lines for better readability. Here is an example +of a multi line block comment: + +``` + #!/usr/bin/env stack + {- stack + --resolver lts-3.2 + --install-ghc + runghc + --package turtle + -} +``` +## Finding project configs, and the implicit global + +Whenever you run something with stack, it needs a stack.yaml project file. The +algorithm stack uses to find this is: + +1. Check for a `--stack-yaml` option on the command line +2. Check for a `STACK_YAML` environment variable +3. Check the current directory and all ancestor directories for a `stack.yaml` + +The first two provide a convenient method for using an alternate configuration. +For example: `stack build --stack-yaml stack-7.8.yaml` can be used by your CI +system to check your code against GHC 7.8. Setting the `STACK_YAML` environment +variable can be convenient if you're going to be running commands like `stack +ghc` in other directories, but you want to use the configuration you defined in +a specific project. + +If stack does not find a stack.yaml in any of the three specified locations, +the *implicit global* logic kicks in. You've probably noticed that phrase a few +times in the output from commands above. Implicit global is essentially a hack +to allow stack to be useful in a non-project setting. When no implicit global +config file exists, stack creates one for you with the latest LTS snapshot as +the resolver. This allows you to do things like: + +* compile individual files easily with `stack ghc` +* build executables without starting a project, e.g. `stack install pandoc` + +Keep in mind that there's nothing magical about this implicit global +configuration. It has no impact on projects at all. Every package you install +with it is put into isolated databases just like everywhere else. The only magic +is that it's the catch-all project whenever you're running stack somewhere else. + +## stack.yaml vs .cabal files + +Now that we've covered a lot of stack use cases, this quick summary of +stack.yaml vs .cabal files will hopefully make sense and be a good reminder for +future uses of stack: + +* A project can have multiple packages. +* Each project has a stack.yaml. +* Each package has a .cabal file. +* The .cabal file specifies which packages are dependencies. +* The stack.yaml file specifies which packages are available to be used. +* .cabal specifies the components, modules, and build flags provided by a package +* stack.yaml can override the flag settings for individual packages +* stack.yaml specifies which packages to include + +## Comparison to other tools + +stack is not the only tool around for building Haskell code. stack came into +existence due to limitations with some of the existing tools. If you're +unaffected by those limitations and are happily building Haskell code, you may +not need stack. If you're suffering from some of the common problems in other +tools, give stack a try instead. + +If you're a new user who has no experience with other tools, we recommend going +with stack. The defaults match modern best practices in Haskell development, and +there are less corner cases you need to be aware of. You *can* develop Haskell +code with other tools, but you probably want to spend your time writing code, +not convincing a tool to do what you want. + +Before jumping into the differences, let me clarify an important similarity: + +__Same package format.__ stack, cabal-install, and presumably all other tools +share the same underlying Cabal package format, consisting of a .cabal file, +modules, etc. This is a Good Thing: we can share the same set of upstream +libraries, and collaboratively work on the same project with stack, +cabal-install, and NixOS. In that sense, we're sharing the same ecosystem. + +Now the differences: + +* __Curation vs dependency solving as a default__. + * stack defaults to using curation (Stackage snapshots, LTS Haskell, + Nightly, etc) as a default instead of defaulting to dependency solving, as + cabal-install does. This is just a default: as described above, stack can + use dependency solving if desired, and cabal-install can use curation. + However, most users will stick to the defaults. The stack team firmly + believes that the majority of users want to simply ignore dependency + resolution nightmares and get a valid build plan from day 1, which is why + we've made this selection of default behavior. +* __Reproducible__. + * stack goes to great lengths to ensure that `stack build` today does the + same thing tomorrow. cabal-install does not: build plans can be affected + by the presence of preinstalled packages, and running `cabal update` can + cause a previously successful build to fail. With stack, changing the + build plan is always an explicit decision. +* __Automatically building dependencies__. + * In cabal-install, you need to use `cabal install` to trigger dependency + building. This is somewhat necessary due to the previous point, since + building dependencies can, in some cases, break existing installed + packages. So for example, in stack, `stack test` does the same job as + `cabal install --run-tests`, though the latter *additionally* performs an + installation that you may not want. The closer command equivalent is + `cabal install --enable-tests --only-dependencies && cabal configure + --enable-tests && cabal build && cabal test` (newer versions of + cabal-install may make this command shorter). +* __Isolated by default__. + * This has been a pain point for new stack users. In cabal, the + default behavior is a non-isolated build where working on two projects can + cause the user package database to become corrupted. The cabal solution to + this is sandboxes. stack, however, provides this behavior by default via + its databases. In other words: when you use stack, there's __no need for + sandboxes__, everything is (essentially) sandboxed by default. + +__Other tools for comparison (including active and historical)__ +* [cabal-dev](https://hackage.haskell.org/package/cabal-dev) (deprecated in favor of cabal-install) +* [cabal-meta](https://hackage.haskell.org/package/cabal-meta) inspired a lot of the multi-package functionality of stack. If you're still using cabal-install, cabal-meta is relevant. For stack work, the feature set is fully subsumed by stack. +* [cabal-src](https://hackage.haskell.org/package/cabal-src) is mostly irrelevant in the presence of both stack and cabal sandboxes, both of which make it easier to add additional package sources easily. The mega-sdist executable that ships with cabal-src is, however, still relevant. Its functionality may some day be folded into stack +* [stackage-cli](https://hackage.haskell.org/package/stackage-cli) was an initial attempt to make cabal-install work more easily with curated snapshots, but due to a slight impedance mismatch between cabal.config constraints and snapshots, it did not work as well as hoped. It is deprecated in favor of stack. + +## More resources + +There are lots of resources available for learning more about stack: + +* `stack --help` +* `stack --version` — identify the version and Git hash of the stack executable +* `--verbose` (or `-v`) — much more info about internal operations (useful for bug reports) +* The [home page](http://haskellstack.org) +* The [stack mailing list](https://groups.google.com/d/forum/haskell-stack) +* The [the FAQ](faq.md) +* The [stack wiki](https://github.com/commercialhaskell/stack/wiki) +* The [haskell-stack tag on Stack Overflow](http://stackoverflow.com/questions/tagged/haskell-stack) +* [Another getting started with stack tutorial](http://seanhess.github.io/2015/08/04/practical-haskell-getting-started.html) +* [Why is stack not cabal?](https://www.fpcomplete.com/blog/2015/06/why-is-stack-not-cabal) + + +## Fun features + +This is just a quick collection of fun and useful feature stack supports. + +### Templates + +We started off using the `new` command to create a project. stack provides +multiple templates to start a new project from: + +``` +michael@d30748af6d3d:~$ stack templates +chrisdone +hakyll-template +new-template +simple +yesod-minimal +yesod-mongo +yesod-mysql +yesod-postgres +yesod-postgres-fay +yesod-simple +yesod-sqlite +michael@d30748af6d3d:~$ stack new my-yesod-project yesod-simple +Downloading template "yesod-simple" to create project "my-yesod-project" in my-yesod-project/ ... +Using the following authorship configuration: +author-email: example@example.com +author-name: Example Author Name +Copy these to /home/michael/.stack/config.yaml and edit to use different values. +Writing default config file to: /home/michael/my-yesod-project/stack.yaml +Basing on cabal files: +- /home/michael/my-yesod-project/my-yesod-project.cabal + +Checking against build plan lts-3.2 +Selected resolver: lts-3.2 +Wrote project config to: /home/michael/my-yesod-project/stack.yaml +``` + +To add more templates, see [the stack-templates +repository](https://github.com/commercialhaskell/stack-templates#readme). + +### IDE + +stack has a work-in-progress suite of editor integrations, to do things like +getting type information in Emacs. For more information, see +[stack-ide](https://github.com/commercialhaskell/stack-ide#readme). + +### Visualizing dependencies + +If you'd like to get some insight into the dependency tree of your packages, you +can use the `stack dot` command and Graphviz. More information is +[available in the Dependency visualization documentation](dependency_visualization.md). + +### Travis with caching + +Many people use Travis CI to test out a project for every Git push. We have [a +document devoted to +Travis](travis_ci.md). However, for +most people, the following example will be sufficient to get started: + +```yaml +# Copy these contents into the root directory of your Github project in a file +# named .travis.yml + +# Use new container infrastructure to enable caching +sudo: false + +# Choose a lightweight base image; we provide our own build tools. +language: c + +# Caching so the next build will be fast too. +cache: + directories: + - $HOME/.ghc + - $HOME/.cabal + - $HOME/.stack + +# The different configurations we want to test. We have BUILD=cabal which uses +# cabal-install, and BUILD=stack which uses Stack. More documentation on each +# of those below. +# +# We set the compiler values here to tell Travis to use a different +# cache file per set of arguments. +# +# If you need to have different apt packages for each combination in the +# matrix, you can use a line such as: +# addons: {apt: {packages: [libfcgi-dev,libgmp-dev]}} +matrix: + include: + # We grab the appropriate GHC and cabal-install versions from hvr's PPA. See: + # https://github.com/hvr/multi-ghc-travis + #- env: BUILD=cabal GHCVER=7.0.4 CABALVER=1.16 HAPPYVER=1.19.5 ALEXVER=3.1.7 + # compiler: ": #GHC 7.0.4" + # addons: {apt: {packages: [cabal-install-1.16,ghc-7.0.4,happy-1.19.5,alex-3.1.7], sources: [hvr-ghc]}} + #- env: BUILD=cabal GHCVER=7.2.2 CABALVER=1.16 HAPPYVER=1.19.5 ALEXVER=3.1.7 + # compiler: ": #GHC 7.2.2" + # addons: {apt: {packages: [cabal-install-1.16,ghc-7.2.2,happy-1.19.5,alex-3.1.7], sources: [hvr-ghc]}} + #- env: BUILD=cabal GHCVER=7.4.2 CABALVER=1.16 HAPPYVER=1.19.5 ALEXVER=3.1.7 + # compiler: ": #GHC 7.4.2" + # addons: {apt: {packages: [cabal-install-1.16,ghc-7.4.2,happy-1.19.5,alex-3.1.7], sources: [hvr-ghc]}} + - env: BUILD=cabal GHCVER=7.6.3 CABALVER=1.16 HAPPYVER=1.19.5 ALEXVER=3.1.7 + compiler: ": #GHC 7.6.3" + addons: {apt: {packages: [cabal-install-1.16,ghc-7.6.3,happy-1.19.5,alex-3.1.7], sources: [hvr-ghc]}} + - env: BUILD=cabal GHCVER=7.8.4 CABALVER=1.18 HAPPYVER=1.19.5 ALEXVER=3.1.7 + compiler: ": #GHC 7.8.4" + addons: {apt: {packages: [cabal-install-1.18,ghc-7.8.4,happy-1.19.5,alex-3.1.7], sources: [hvr-ghc]}} + - env: BUILD=cabal GHCVER=7.10.3 CABALVER=1.22 HAPPYVER=1.19.5 ALEXVER=3.1.7 + compiler: ": #GHC 7.10.3" + addons: {apt: {packages: [cabal-install-1.22,ghc-7.10.3,happy-1.19.5,alex-3.1.7], sources: [hvr-ghc]}} + + # Build with the newest GHC and cabal-install. This is an accepted failure, + # see below. + - env: BUILD=cabal GHCVER=head CABALVER=head HAPPYVER=1.19.5 ALEXVER=3.1.7 + compiler: ": #GHC HEAD" + addons: {apt: {packages: [cabal-install-head,ghc-head,happy-1.19.5,alex-3.1.7], sources: [hvr-ghc]}} + + # The Stack builds. We can pass in arbitrary Stack arguments via the ARGS + # variable, such as using --stack-yaml to point to a different file. + - env: BUILD=stack ARGS="" + compiler: ": #stack default" + addons: {apt: {packages: [ghc-7.10.3], sources: [hvr-ghc]}} + + - env: BUILD=stack ARGS="--resolver lts-2" + compiler: ": #stack 7.8.4" + addons: {apt: {packages: [ghc-7.8.4], sources: [hvr-ghc]}} + + - env: BUILD=stack ARGS="--resolver lts-3" + compiler: ": #stack 7.10.2" + addons: {apt: {packages: [ghc-7.10.2], sources: [hvr-ghc]}} + + - env: BUILD=stack ARGS="--resolver lts-5" + compiler: ": #stack 7.10.3" + addons: {apt: {packages: [ghc-7.10.3], sources: [hvr-ghc]}} + + # Nightly builds are allowed to fail + - env: BUILD=stack ARGS="--resolver nightly" + compiler: ": #stack nightly" + addons: {apt: {packages: [libgmp,libgmp-dev]}} + + # Build on OS X in addition to Linux + - env: BUILD=stack ARGS="" + compiler: ": #stack default osx" + os: osx + + - env: BUILD=stack ARGS="--resolver lts-2" + compiler: ": #stack 7.8.4 osx" + os: osx + + - env: BUILD=stack ARGS="--resolver lts-3" + compiler: ": #stack 7.10.2 osx" + os: osx + + - env: BUILD=stack ARGS="--resolver lts-5" + compiler: ": #stack 7.10.3 osx" + os: osx + + - env: BUILD=stack ARGS="--resolver nightly" + compiler: ": #stack nightly osx" + os: osx + + allow_failures: + - env: BUILD=cabal GHCVER=head CABALVER=head HAPPYVER=1.19.5 ALEXVER=3.1.7 + - env: BUILD=stack ARGS="--resolver nightly" + +before_install: +# Using compiler above sets CC to an invalid value, so unset it +- unset CC + +# We want to always allow newer versions of packages when building on GHC HEAD +- CABALARGS="" +- if [ "x$GHCVER" = "xhead" ]; then CABALARGS=--allow-newer; fi + +# Download and unpack the stack executable +- export PATH=/opt/ghc/$GHCVER/bin:/opt/cabal/$CABALVER/bin:$HOME/.local/bin:/opt/alex/$ALEXVER/bin:/opt/happy/$HAPPYVER/bin:$HOME/.cabal/bin:$PATH +- mkdir -p ~/.local/bin +- | + if [ `uname` = "Darwin" ] + then + travis_retry curl --insecure -L https://www.stackage.org/stack/osx-x86_64 | tar xz --strip-components=1 --include '*/stack' -C ~/.local/bin + else + travis_retry curl -L https://www.stackage.org/stack/linux-x86_64 | tar xz --wildcards --strip-components=1 -C ~/.local/bin '*/stack' + fi + + # Use the more reliable S3 mirror of Hackage + mkdir -p $HOME/.cabal + echo 'remote-repo: hackage.haskell.org:http://hackage.fpcomplete.com/' > $HOME/.cabal/config + echo 'remote-repo-cache: $HOME/.cabal/packages' >> $HOME/.cabal/config + + if [ "$CABALVER" != "1.16" ] + then + echo 'jobs: $ncpus' >> $HOME/.cabal/config + fi + +# Get the list of packages from the stack.yaml file +- PACKAGES=$(stack --install-ghc query locals | grep '^ *path' | sed 's@^ *path:@@') + +install: +- echo "$(ghc --version) [$(ghc --print-project-git-commit-id 2> /dev/null || echo '?')]" +- if [ -f configure.ac ]; then autoreconf -i; fi +- | + set -ex + case "$BUILD" in + stack) + stack --no-terminal --install-ghc $ARGS test --bench --only-dependencies + ;; + cabal) + cabal --version + travis_retry cabal update + cabal install --only-dependencies --enable-tests --enable-benchmarks --force-reinstalls --ghc-options=-O0 --reorder-goals --max-backjumps=-1 $CABALARGS $PACKAGES + ;; + esac + set +ex + +script: +- | + set -ex + case "$BUILD" in + stack) + stack --no-terminal $ARGS test --bench --no-run-benchmarks --haddock --no-haddock-deps + ;; + cabal) + cabal install --enable-tests --enable-benchmarks --force-reinstalls --ghc-options=-O0 --reorder-goals --max-backjumps=-1 $CABALARGS $PACKAGES + + ORIGDIR=$(pwd) + for dir in $PACKAGES + do + cd $dir + cabal check || [ "$CABALVER" == "1.16" ] + cabal sdist + SRC_TGZ=$(cabal info . | awk '{print $2;exit}').tar.gz && \ + (cd dist && cabal install --force-reinstalls "$SRC_TGZ") + cd $ORIGDIR + done + ;; + esac + set +ex +``` + +Not only will this build and test your project against multiple GHC versions +and snapshots, but it will cache your snapshot built packages, meaning that +subsequent builds will be much faster. + +Once Travis whitelists the stack .deb files, we'll be able to simply include +stack in the `addons` section, and automatically use the newest version of +stack, avoiding that complicated `before_install` section This is being +tracked in the +[apt-source-whitelist](https://github.com/travis-ci/apt-source-whitelist/pull/7) +and +[apt-package-whitelist](https://github.com/travis-ci/apt-package-whitelist/issues/379) +issue trackers. + +In case you're wondering: we need `--no-terminal` because stack does some fancy +sticky display on smart terminals to give nicer status and progress messages, +and the terminal detection is broken on Travis. + +### Shell auto-completion + +Love tab-completion of commands? You're not alone. If you're on bash, just run +the following (or add it to `.bashrc`): + + eval "$(stack --bash-completion-script stack)" + +For more information and other shells, see [the Shell auto-completion wiki +page](https://github.com/commercialhaskell/stack/wiki/Shell-autocompletion) + +### Docker + +stack provides two built-in Docker integrations. Firstly, you can build your +code inside a Docker image, which means: + +* even more reproducibility to your builds, since you and the rest of your team + will always have the same system libraries +* the Docker images ship with entire precompiled snapshots. That means you have + a large initial download, but much faster builds + +For more information, see +[the Docker-integration documentation](docker_integration.md). + +stack can also generate Docker images for you containing your built executables. +This feature is great for automating deployments from CI. This feature is not +yet well-documented, but the basics are to add a section like the following +to stack.yaml: + +```yaml +image: + # YOU NEED A `container` YAML SECTION FOR `stack image container` + container: + # YOU NEED A BASE IMAGE NAME. STACK LAYERS EXES ON TOP OF + # THE BASE IMAGE. PREPARE YOUR PROJECT IMAGE IN ADVANCE. PUT + # ALL YOUR RUNTIME DEPENDENCIES IN THE IMAGE. + base: "fpco/ubuntu-with-libgmp:14.04" + # YOU CAN OPTIONALY NAME THE IMAGE. STACK WILL USE THE PROJECT + # DIRECTORY NAME IF YOU LEAVE OUT THIS OPTION. + name: "fpco/hello-world" + # OPTIONALLY ADD A HASH OF LOCAL PROJECT DIRECTORIES AND THEIR + # DESTINATIONS INSIDE THE DOCKER IMAGE. + add: + man/: /usr/local/share/man/ + # OPTIONALLY SPECIFY A LIST OF EXECUTABLES. STACK WILL CREATE + # A TAGGED IMAGE FOR EACH IN THE LIST. THESE IMAGES WILL HAVE + # THEIR RESPECTIVE "ENTRYPOINT" SET. + entrypoints: + - stack +``` + +and then run `stack image container` and then `docker images` to list +the images. + +### Nix + +stack provides an integration with [Nix](http://nixos.org/nix), +providing you with the same two benefits as the first Docker +integration discussed above: + +* more reproducible builds, since fixed versions of any system + libraries and commands required to build the project are + automatically built using Nix and managed locally per-project. These + system packages never conflict with any existing versions of these + libraries on your system. That they are managed locally to the + project means that you don't need to alter your system in any way to + build any odd project pulled from the Internet. +* implicit sharing of system packages between projects, so you don't + have more copies on-disk than you need to. + +When using the Nix integration, Stack downloads and builds Haskell dependencies +as usual, but resorts on Nix to provide non-Haskell dependencies that exist in +the Nixpkgs. + +Both Docker and Nix are methods to *isolate* builds and thereby make +them more reproducible. They just differ in the means of achieving +this isolation. Nix provides slightly weaker isolation guarantees than +Docker, but is more lightweight and more portable (Linux and OS +X mainly, but also Windows). For more on Nix, its command-line +interface and its package description language, read the +[Nix manual](http://nixos.org/nix/manual). But keep in mind that the +point of stack's support is to obviate the need to write any Nix code +in the common case or even to learn how to use the Nix tools (they're +called under the hood). + +For more information, see +[the Nix-integration documentation](nix_integration.md). + +## Power user commands + +The following commands are a little more powerful, and won't be needed by all +users. Here's a quick rundown: + +* `stack update` will download the most recent set of packages from your package + indices (e.g. Hackage). Generally, stack runs this for you automatically + when necessary, but it can be useful to do this manually sometimes (e.g., + before running `stack solver`, to guarantee you have the most recent + upstream packages available). +* `stack unpack` is a command we've already used quite a bit for examples, but + most users won't use it regularly. It does what you'd expect: downloads a + tarball and unpacks it. +* `stack sdist` generates an uploading tarball containing your package code +* `stack upload` uploads an sdist to Hackage. In the future, it will also + perform automatic GPG signing of your packages for additional security, when + configured. + * `--sign` provides a way to GPG sign your package & submit the result to + sig.commercialhaskell.org for storage in the sig-archive git + repo. (Signatures will be used later to verify package integrity.) +* `stack upgrade` will build a new version of stack from source. + * `--git` is a convenient way to get the most recent version from master for + those testing and living on the bleeding edge. +* `stack setup --upgrade-cabal` can install a newer version of the Cabal + library, used for performing actual builds. You shouldn't generally do this, + since new Cabal versions may introduce incompatibilities with package sets, + but it can be useful if you're trying to test a specific bugfix. +* `stack list-dependencies` lists all of the packages and versions used for a + project +* `stack sig` subcommand can help you with GPG signing & verification + * `sign` will sign an sdist tarball and submit the signature to + sig.commercialhaskell.org for storage in the sig-archive git repo. + (Signatures will be used later to verify package integrity.) + +## Debugging + +The following command installs with profiling enabled: + +`stack install --enable-executable-profiling --enable-library-profiling +--ghc-options="-rtsopts"` + +This command will allow you to use various tools to profile the time, +allocation, heap, and more of a program. The `-prof` GHC option is unnecessary +and will result in a warning. Additional compilation options can be added to +`--ghc-options` if needed. To see a general overview of the time and allocation +of a program called `main` compiled with the above command, you can run + +`./main +RTS -p` + +to generate a `main.prof` file containing the requested profiling information. +Alternatively, you can use `stack exec main -- +RTS -p`, where `--` allows you +to specify parameters for `main` executable (without `--` the `stack` +executable would use those parameters instead). + + +For more commands and uses, see [the official GHC chapter on +profiling](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/profiling.html), +[the Haskell wiki](https://wiki.haskell.org/How_to_profile_a_Haskell_program), +and [the chapter on profiling in Real World +Haskell](http://book.realworldhaskell.org/read/profiling-and-optimization.html). diff --git a/doc/MAINTAINER_GUIDE.md b/doc/MAINTAINER_GUIDE.md new file mode 100644 --- /dev/null +++ b/doc/MAINTAINER_GUIDE.md @@ -0,0 +1,138 @@ +# Maintainer guide + +## Next release: + +* Check that GPG signature for + [Releases](https://s3.amazonaws.com/download.fpcomplete.com/debian/dists/jessie/Release) + uses SHA512, and close + [#1943](https://github.com/commercialhaskell/stack/issues/1934) +* Integrate FreeBSD binaries and packages + [#1253](https://github.com/commercialhaskell/stack/issues/1253#issuecomment-185993240) + +## Pre-release checks + +The following should be tested minimally before a release is considered good +to go: + +* Ensure `release` and `stable` branches merged to `master` +* Integration tests pass on a representative Windows, Mac OS X, and Linux (Linux + is handled by Jenkins automatically): `stack install --pedantic && stack test + --pedantic --flag stack:integration-tests` . The actual release script will + perform a more thorough test for every platform/variant prior to uploading, so + this is just a pre-check +* Ensure `stack haddock` works (Travis CI now does this) +* Stack builds with `stack-7.8.yaml` (Travis CI now does this) +* stack can build the wai repo +* Running `stack build` a second time on either stack or wai is a no-op +* Build something that depends on `happy` (suggestion: `hlint`), since `happy` + has special logic for moving around the `dist` directory +* In master branch: + * stack.cabal: bump the version number to release (even third + component) + * ChangeLog: rename the "unreleased changes" section to the new version +* Cut a release candidate branch `rc/vX.Y.Z` from master +* In master branch: + * stack.cabal: bump version number to unstable (odd third component) + * Changelog: add new "unreleased changes" section + * stack.yaml: bump to use latest LTS version, and check whether extra-deps + still needed +* In RC branch: + * Update the ChangeLog + ([this comparison](https://github.com/commercialhaskell/stack/compare/release...master) + is handy): + * Check for any important changes that missed getting an entry in Changelog + * Check for any entries that snuck into the previous version's changes + due to merges + * Review documentation for any changes that need to be made + * Search for old Stack version, unstable stack version, and the next + "obvious" version in sequence (if doing a non-obvious jump) and replace + with new version + * Look for any links to "latest" documentation, replace with version tag + * Ensure all documentation pages listed in `mkdocs.yaml` + * Check that any new Linux distribution versions added to + `etc/scripts/release.hs` and `etc/scripts/vagrant-releases.sh`. + * [Ubuntu](https://wiki.ubuntu.com/Releases) + * [Debian](https://www.debian.org/releases/) (keep at least latest two) + * [CentOS](https://wiki.centos.org/Download) + * [Fedora](https://fedoraproject.org/wiki/Releases) + * Check that no new entries need to be added to + [releases.yaml](https://github.com/fpco/stackage-content/blob/master/stack/releases.yaml), + [install_and_upgrade.md](https://github.com/commercialhaskell/stack/blob/master/doc/install_and_upgrade.md), + and + `README.md` + * Remove unsupported/obsolete distribution versions from + [install_and_upgrade.md](https://github.com/commercialhaskell/stack/blob/master/doc/install_and_upgrade.md), + and perhaps from the release process. + +## Release process + +See +[stack-release-script's README](https://github.com/commercialhaskell/stack/blob/master/etc/scripts/README.md#prerequisites) +for requirements to perform the release, and more details about the tool. + +* Create a + [new draft Github release](https://github.com/commercialhaskell/stack/releases/new) + with tag and name `vX.Y.Z` (where X.Y.Z is the stack package's version), targetting the + RC branch + +* On each machine you'll be releasing from, set environment variables: + `GITHUB_AUTHORIZATION_TOKEN`, `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, + `AWS_DEFAULT_REGION`. + + Note: since one of the tools (rpm-s3 on CentOS) doesn't support AWS temporary + credentials, you can't use MFA with the AWS credentials (`AWS_SECURITY_TOKEN` + is ignored). + +* On a machine with Vagrant installed: + * Run `etc/scripts/vagrant-releases.sh` + +* On Mac OS X: + * Run `etc/scripts/osx-release.sh` + +* On Windows: + * Ensure your working tree is in `C:\stack` (or a similarly short path) + * Run `etc\scripts\windows-releases.bat` + * Release Windows installers. See + [stack-installer README](https://github.com/borsboom/stack-installer#readme) + +* Push signed Git tag, matching Github release tag name, e.g.: `git tag -u + 0x575159689BEFB442 vX.Y.Z && git push origin vX.Y.Z` + +* Reset the `release` branch to the released commit, e.g.: `git checkout release + && git merge --ff-only vX.Y.Z && git push origin release` + +* Update the `stable` branch similarly + +* Delete the RC branch (locally and on origin) + +* Publish Github release + +* Edit + [stack-setup-2.yaml](https://github.com/fpco/stackage-content/blob/master/stack/stack-setup-2.yaml), + and add the new linux64 stack bindist + +* Activate version for new release tag on + [readthedocs.org](https://readthedocs.org/projects/stack/versions/), and + ensure that stable documentation has updated + +* Upload package to Hackage: `stack upload . --pvp-bounds=both` + +* On a machine with Vagrant installed: + * Run `etc/scripts/vagrant-distros.sh` + +* Submit a PR for the + [haskell-stack Homebrew formula](https://github.com/Homebrew/homebrew/blob/master/Library/Formula/haskell-stack.rb) + * Be sure to update the SHA sum + * The commit message should just be `haskell-stack ` + +* [Flag the Arch Linux package as out-of-date](https://www.archlinux.org/packages/community/x86_64/stack/flag/) + +* Upload haddocks to Hackage: `etc/scripts/upload-haddocks.sh` + +* Merge any changes made in the RC/release/stable branches to master. + +* Announce to haskell-cafe@haskell.org haskell-stack@googlegroups.com + commercialhaskell@googlegroups.com mailing lists + +* Keep an eye on the + [Hackage matrix builder](http://matrix.hackage.haskell.org/package/stack) diff --git a/doc/README.md b/doc/README.md new file mode 100644 --- /dev/null +++ b/doc/README.md @@ -0,0 +1,172 @@ +# The Haskell Tool Stack + +Stack is a cross-platform program for developing Haskell +projects. It is aimed at Haskellers both new and experienced. + + + +It features: + +* Installing GHC automatically, in an isolated location. +* Installing packages needed for your project. +* Building your project. +* Testing your project. +* Benchmarking your project. + +#### How to install + +Downloads are available by operating system: + +* [Windows](install_and_upgrade.md#windows) +* [Mac OS X](install_and_upgrade.md#mac-os-x) +* [Ubuntu](install_and_upgrade.md#ubuntu) +* [Debian](install_and_upgrade.md#debian) +* [CentOS / Red Hat / Amazon Linux](install_and_upgrade.md#centos) +* [Fedora](install_and_upgrade.md#fedora) +* [openSUSE / SUSE Linux Enterprise](install_and_upgrade.md#suse) +* [Arch Linux](install_and_upgrade.md#arch-linux) +* [NixOS](install_and_upgrade.md#nixos) +* [Linux (general)](install_and_upgrade.md#linux) +* [FreeBSD (unofficial)](install_and_upgrade.md#freebsd) + +[Upgrade instructions](install_and_upgrade.md#upgrade) + +Note: if you are using cabal-install to install stack, you may need to pass a +constraint to work around a +[Cabal issue](https://github.com/haskell/cabal/issues/2759): `cabal install +--constraint 'mono-traversable >= 0.9' stack`. + +#### Quick Start Guide + +First you need to [install it (see previous section)](#how-to-install). + +##### Start your new project: + +```bash +stack new my-project +cd my-project +stack setup +stack build +stack exec my-project-exe +``` + +- The `stack new` command will create a new directory containing all +the needed files to start a project correctly. +- The `stack setup` will download the compiler if necessary in an isolated + location (default `~/.stack`) that won't interfere with any system-level + installations. (For information on installation paths, please use the `stack + path` command.). +- The `stack build` command will build the minimal project. +- `stack exec my-project-exe` will execute the command. +- If you just want to install an executable using stack, then all you have to do +is`stack install `. + +If you want to launch a REPL: + +```bash +stack ghci +``` + + +Run `stack` for a complete list of commands. + +##### Workflow + +The `stack new` command should have created the following files: + +``` +. +├── LICENSE +├── Setup.hs +├── app +│   └── Main.hs +├── my-project.cabal +├── src +│   └── Lib.hs +├── stack.yaml +└── test + └── Spec.hs + + 3 directories, 7 files +``` + +So to manage your library: + +1. Edit files in the `src/` directory. + +The `app` directory should preferably contain only files related to +executables. + +2. If you need to include another library (for example the package [`text`](https://hackage.haskell.org/package/text): + + - Add the package `text` to the file `my-project.cabal` + in the section `build-depends: ...`. + - run `stack build` another time + +3. If you get an error that tells you your package isn't in the LTS. + Just try to add a new version in the `stack.yaml` file in the `extra-deps` section. + +It was a really fast introduction on how to start to code in Haskell using `stack`. +If you want to go further, we highly recommend you to read the [`stack` guide](GUIDE.md). + +#### How to contribute + +This assumes that you have already installed a version of stack, and have `git` +installed. + +1. Clone `stack` from git with + `git clone https://github.com/commercialhaskell/stack.git`. +2. Enter into the stack folder with `cd stack`. +3. Build `stack` using a pre-existing `stack` install with + `stack setup && stack build`. +4. Once `stack` finishes building, check the stack version with + `stack exec stack -- --version`. Make sure the version is the latest. +5. Look for issues tagged with + [`newcomer` and `awaiting-pr` labels](https://github.com/commercialhaskell/stack/issues?q=is%3Aopen+is%3Aissue+label%3Anewcomer+label%3A%22awaiting+pr%22). + +Build from source as a one-liner: + +```bash +git clone https://github.com/commercialhaskell/stack.git && \ +cd stack && \ +stack setup && \ +stack build +``` + +#### Complete guide to stack + +This repository also contains a complete [user guide to using stack +](GUIDE.md), covering all of the most common use cases. + + +#### Questions, Feedback, Discussion + +* For frequently asked questions about detailed or specific use-cases, please + see [the FAQ](faq.md). +* For general questions, comments, feedback and support please write + to [the stack mailing list](https://groups.google.com/d/forum/haskell-stack). +* For bugs, issues, or requests please + [open an issue](https://github.com/commercialhaskell/stack/issues/new). +* When using Stack Overflow, please use [the haskell-stack + tag](http://stackoverflow.com/questions/tagged/haskell-stack). + +#### Why stack? + +stack is a project of the [Commercial Haskell](http://commercialhaskell.com/) +group, spearheaded by [FP Complete](https://www.fpcomplete.com/). It is +designed to answer the needs of commercial Haskell users, hobbyist Haskellers, +and individuals and companies thinking about starting to use Haskell. It is +intended to be easy to use for newcomers, while providing the customizability +and power experienced developers need. + +While stack itself has been around since June of 2015, it is based on codebases +used by FP Complete for its corporate customers and internally for years prior. +stack is a refresh of that codebase combined with other open source efforts +like [stackage-cli](https://github.com/fpco/stackage-cli) to meet the needs of +users everywhere. + +A large impetus for the work on stack was a [large survey of people interested +in +Haskell](https://www.fpcomplete.com/blog/2015/05/thousand-user-haskell-survey), +which rated build issues as a major concern. The stack team hopes that stack +can address these concerns. diff --git a/doc/SIGNING_KEY.md b/doc/SIGNING_KEY.md new file mode 100644 --- /dev/null +++ b/doc/SIGNING_KEY.md @@ -0,0 +1,36 @@ +# Signing key + +Releases are signed with this key: + +``` +-----BEGIN PGP PUBLIC KEY BLOCK----- +Version: GnuPG v1 + +mQENBFVs+cMBCAC5IsLWTikd1V70Ur1FPJMn14Sc/C2fbXc0zRcPuWX+JaXgrIJQ +74A3UGBpa07wJDZiQLLz4AasDQj++9gXdiM9MlK/xWt8BQpgQqSMgkktFVajSWX2 +rSXPjqLtsl5dLsc8ziBkd/AARXoeITmXX+n6oRTy6QfdMv2Tacnq7r9M9J6bAz6/ +7UsKkyZVwsbUPea4SuD/s7jkXAuly15APaYDmF5mMlpoRWp442lJFpA0h52mREX1 +s5FDbuKRQW7OpZdLcmOgoknJBDSpKHuHEoUhdG7Y3WDUGYFZcTtta1qSVHrm3nYa +7q5yOzPW4/VpftkBs1KzIxx0nQ5INT5W5+oTABEBAAG0H0ZQQ29tcGxldGUgPGRl +dkBmcGNvbXBsZXRlLmNvbT6JATcEEwEKACEFAlVs+cMCGwMFCwkIBwMFFQoJCAsF +FgMCAQACHgECF4AACgkQV1FZaJvvtEIP8gf/S/k4C3lp/BFb0K9DHHSt6EaGQPwy +g+O8d+JvL7ghkvMjlQ+UxDw+LfRKANTpl8a4vHtEQLHEy1tPJfrnMA8DNci8HLVx +rK3lIqMfv5t85VST9rz3X8huSw7qwFyxsmIqFtJC/BBQfsOXC+Q5Z2nbResXHMeA +5ZvDopZnqKPdmMOngabPGZd89hOKn6r8k7+yvZ/mXmrGOB8q5ZGbOXUbCshst7lc +yZWmoK3VJdErQjGHCdF4MC9KFBQsYYUy9b1q0OUv9QLtq/TeKxfpvYk9zMWAoafk +M8QBE/qqOpqkBRoKbQHCDQgx7AXJMKnOA0jPx1At57hWl7PuEH4rK38UtLkBDQRV +bPnDAQgAx1+4ENyaMk8XznQQ4l+nl8qw4UedZhnR5Xxr6z2kcMO/0VdwmIDCpxaM +spurOF+yExfY/Chbex7fThWTwVgfsItUc/QLLv9jkvpveMUDuPyh/4QrAQBYoW09 +jMJcOTFQU+f4CtKaN/1PNoTSU2YkVpbhvtV3Jn2LPFjUSPb7z2NZ9NKe10M0/yN+ +l0CuPlqu6GZR5L3pA5i8PZ0Nh47j0Ux5KIjrjCGne4p+J8qqeRhUf04yHAYfDLgE +aLAG4v4pYbb1jNPUm1Kbk0lo2c3dxx0IU201uAQ6LNLdF/WW/ZF7w3iHn7kbbzXO +jhbq2rvZEn3K9xDr7homVnnj21/LSQARAQABiQEfBBgBCgAJBQJVbPnDAhsMAAoJ +EFdRWWib77RC3ukH/R9jQ4q6LpXynQPJJ9QKwstglKfoKNpGeAYVTEn0e7NB0HV5 +BC+Da5SzBowboxC2YCD1wTAjBjLLQfAYNyR+tHpJBaBmruafj87nBCDhSWwWDXwx +OUDpNOwKUkrwZDRlM7n4byoMRl7Vh/7CXxaTqkyao1c5v3mHh/DremiTvOJ4OXgJ +77NHaPXezHkCFZC8/sX6aY0DJxF+LIE84CoLI1LYBatH+NKxoICKA+yeF3RIVw0/ +F3mtEFEtmJ6ljSks5tECxfJFvQlkpILBbGvHfuljKMeaj+iN+bsHmV4em/ELB1ku +N9Obs/bFDBMmQklIdLP7dOunDjY4FwwcFcXdNyg= +=YUsC +-----END PGP PUBLIC KEY BLOCK----- +``` diff --git a/doc/architecture.md b/doc/architecture.md new file mode 100644 --- /dev/null +++ b/doc/architecture.md @@ -0,0 +1,133 @@ +# Architecture + +## Terminology + +* Package identifier: a package name and version, e.g. text-1.2.1.0 +* GhcPkgId: a package identifier plus the unique hash for the generated binary, e.g. text-1.2.1.0-bb83023b42179dd898ebe815ada112c2 +* Package index: a collection of packages available for download. This is a combination of an index containing all of the .cabal files (either a tarball downloaded via HTTP(S) or a Git repository) and some way to download package tarballs. + * By default, stack uses a single package index (the Github/S3 mirrors of Hackage), but supports customization and adding more than one index +* Package database: a collection of metadata about built libraries +* Install root: a destination for installing packages into. Contains a bin path (for generated executables), lib (for the compiled libraries), pkgdb (for the package database), and a few other things +* Snapshot: an LTS Haskell or Stackage Nightly, which gives information on a complete set of packages. This contains a lot of metadata, but importantly it can be converted into a mini build plan... +* Mini build plan: a collection of package identifiers and their build flags that are known to build together +* Resolver: the means by which stack resolves dependencies for your packages. The two currently supported options are snapshot (using LTS or Nightly), and GHC (which installs no extra dependencies). Others may be added in the future (such as a SAT-based dependency solver). These packages are always taken from a package index +* extra-deps: additional packages to be taken from the package index for dependencies. This list will *shadow* packages provided by the resolver +* Local packages: source code actually present on your file system, and referred to by the `packages` field in your stack.yaml file. Each local package has exactly one .cabal file +* Project: a stack.yaml config file and all of the local packages it refers to. + +## Databases + +Every build uses three distinct install roots, which means three separate package databases and bin paths. These are: + +* Global: the packages that ship with GHC. We never install anything into this database +* Snapshot: a database shared by all projects using the same snapshot. Packages installed in this database must use the exact same dependencies and build flags as specified in the snapshot, and cannot be affected by user flags, ensuring that one project cannot corrupt another. There are two caveats to this: + * If different projects use different package indices, then their definitions of what package foo-1.2.3 are may be different, in which case they *can* corrupt each other's shared databases. This is warned about in the FAQ + * Turning on profiling may cause a package to be recompiled, which will result in a different GhcPkgId +* Local: extra-deps, local packages, and snapshot packages which depend on them (more on that in shadowing) + +## Building + +### Shadowing + +Every project must have precisely one version of a package. If one of your +local packages or extra dependencies conflicts with a package in the snapshot, +the local/extradep *shadows* the snapshot version. The way this works is: + +* The package is removed from the list of packages in the snapshot +* Any package that depends on that package (directly or indirectly) is moved from the snapshot to extra-deps, so that it is available to your packages as dependencies. + * Note that there is no longer any guarantee that this package will build, since you're using an untested dependency + +After shadowing, you end up with what is called internally a `SourceMap`, which +is `Map PackageName PackageSource`, where a `PackageSource` can be either a +local package, or a package taken from a package index (specified as a version +number and the build flags). + +### Installed packages + +Once you have a `SourceMap`, you can inspect your three available databases and +decide which of the installed packages you wish to use from them. We move from +the global, to snapshot, and finally local, with the following rules: + +* If we require profiling, and the library does not provide profiling, do not use it +* If the package is in the `SourceMap`, but belongs to a difference database, or has a different version, do not use it +* If after the above two steps, any of the dependencies are unavailable, do not use it +* Otherwise: include the package in the list of installed packages + +We do something similar for executables, but maintain our own database of +installed executables, since GHC does not track them for us. + +### Plan construction + +When running a build, we know which packages we want installed (inventively +called "wanteds"), which packages are available to install, and which are +already installed. In plan construction, we put them information together to +decide which packages must be built. The code in Stack.Build.ConstructPlan is +authoritative on this and should be consulted. The basic idea though is: + +* If any of the dependencies have changed, reconfigure and rebuild +* If a local package has any files changed, rebuild (but don't bother reconfiguring) +* If a local package is wanted and we're running tests or benchmarks, run the test or benchmark even if the code and dependencies haven't changed + +### Plan execution + +Once we have the plan, execution is a relatively simple process of calling +`runghc Setup.hs` in the correct order with the correct parameters. See +Stack.Build.Execute for more information. + +## Configuration + +stack has two layers of configuration: project and non-project. All of these +are stored in stack.yaml files, but the former has extra fields (resolver, +packages, extra-deps, and flags). The latter can be monoidally combined so that +a system config file provides defaults, which a user can override with +`~/.stack/config.yaml`, and a project can further customize. In addition, +environment variables STACK\_ROOT and STACK\_YAML can be used to tweak where +stack gets its configuration from. + +stack follows a simple algorithm for finding your project configuration file: +start in the current directory, and keep going to the parent until it finds a +`stack.yaml`. When using `stack ghc` or `stack exec` as mentioned above, you'll +sometimes want to override that behavior and point to a specific project in +order to use its databases and bin directories. To do so, simply set the +`STACK_YAML` environment variable to point to the relevant `stack.yaml` file. + +## Snapshot auto-detection + +When you run `stack build` with no stack.yaml, it will create a basic +configuration with a single package (the current directory) and an +auto-detected snapshot. The algorithm it uses for selecting this snapshot is: + +* Try the latest two LTS major versions at their most recent minor version release, and the most recent Stackage Nightly. For example, at the time of writing, this would be lts-2.10, lts-1.15, and nightly-2015-05-26 +* For each of these, test the version bounds in the package's .cabal file to see if they are compatible with the snapshot, choosing the first one that matches +* If no snapshot matches, uses the most recent LTS snapshot, even though it will not compile + +If you end up in the no compatible snapshot case, you typically have three options to fix things: + +* Manually specify a different snapshot that you know to be compatible. If you can do that, great, but typically if the auto-detection fails, it means that there's no compatible snapshot +* Modify version bounds in your .cabal file to be compatible with the selected snapshot +* Add `extra-deps` to your stack.yaml file to fix compatibility problems + +Remember that running `stack build` will give you information on why your build +cannot occur, which should help guide you through the steps necessary for the +second and third option above. Also, note that those options can be +mixed-and-matched, e.g. you may decide to relax some version bounds in your +.cabal file, while also adding some extra-deps. + +## Explicit breakage + +As mentioned above, updating your package indices will not cause stack to +invalidate any existing package databases. That's because stack is always +explicit about build plans, via: + +1. the selected snapshot +2. the extra-deps +3. local packages + +The only way to change a plan for packages to be installed is by modifying one +of the above. This means that breakage of a set of installed packages is an +*explicit* and *contained* activity. Specifically, you get the following +guarantees: + +* Since snapshots are immutable, the snapshot package database will not be invalidated by any action. If you change the snapshot you're using, however, you may need to build those packages from scratch. +* If you modify your extra-deps, stack may need to unregister and reinstall them. +* Any changes to your local packages trigger a rebuild of that package and its dependencies. diff --git a/doc/build_command.md b/doc/build_command.md new file mode 100644 --- /dev/null +++ b/doc/build_command.md @@ -0,0 +1,116 @@ +# Build command + +## Overview + +The primary command you use in stack is build. This page describes the details +of build's command line interface. The goal of the interface is to do the right +thing for simple input, and allow a lot of flexibility for more complicated +goals. + +## Synonyms + +One potential point of confusion is the synonym commands for `build`. These are +provided to match commonly expected command line interfaces, and to make common +workflows shorter. The important thing to note is that all of these are just +the `build` command in disguise. Each of these commands are called out as +synonyms in the `--help` output. These commands are: + +* `stack test` is the same as `stack build --test` +* `stack bench` is the same as `stack build --bench` +* `stack haddock` is the same as `stack build --haddock` +* `stack install` is the same as `stack build --copy-bins` + +The advantage of the synonym commands is that they're convenient and short. The +advantage of the options is that they compose. For example, `stack build --test +--copy-bins` will build libraries, executables, and test suites, run the test +suites, and then copy the executables to your local bin path (more on this +below). + +## Components + +Components are a subtle yet important point to how build operates under the +surface. Every cabal package is made up of one or more components. It can have +0 or 1 libraries, and then 0 or more of executable, test, and benchmark +components. stack allows you to call out a specific component to be built, e.g. +`stack build mypackage:test:mytests` will build the `mytests` component of the +`mypackage` package. `mytests` must be a test suite component. + +We'll get into the details of the target syntax for how to select components in +the next section. In this section, the important point is: whenever you target +a test suite or a benchmark, it's built __and also run__, unless you explicitly +disable running via `--no-run-tests` or `--no-run-benchmarks`. Case in point: +the previous command will in fact build the `mytests` test suite *and* run it, +even though you haven't used the `stack test` command or the `--test` option. +(We'll get to what exactly `--test` does below.) + +This gives you a lot of flexibility in choosing what you want stack to do. You +can run a single test component from a package, run a test component from one +package and a benchmark from another package, etc. + +One final note on components: you can only control components for local +packages, not dependencies. With dependencies, stack will *always* build the +library (if present) and all executables, and ignore test suites and +benchmarks. If you want more control over a package, you must add it to your +`packages` setting in your stack.yaml file. + +## Target syntax + +In addition to a number of options (like the aforementioned `--test`), `stack +build` takes a list of zero or more *targets* to be built. There are a number +of different syntaxes supported for this list: + +* *package*, e.g. `stack build foobar`, is the most commonly used target. It will try to find the package in the following locations: local packages, extra dependencies, snapshots, and package index (e.g. Hackage). If it's found in the package index, then the latest version of that package from the index is implicitly added to your extra dependencies. + + This is where the `--test` and `--bench` flags come into play. If the package is a local package, then all of the test suite and benchmark components are selected to be built, respectively. In any event, the library and executable components are also selected to be built. + +* *package identifier*, e.g. `stack build foobar-1.2.3`, is usually used to include specific package versions from the index. If the version selected conflicts with an existing local package or extra dep, then stack fails with an error. Otherwise, this is the same as calling `stack build foobar`, except instead of using the latest version from the index, the version specified is used. + +* *component*. Instead of referring to an entire package and letting stack decide which components to build, you select individual components from inside a package. This can be done for more fine-grained control over which test suites to run, or to have a faster compilation cycle. There are multiple ways to refer to a specific component (provided for convenience): + + * `packagename:comptype:compname` is the most explicit. The available comptypes are `exe`, `test`, and `bench`. + * `packagename:compname` allows you to leave off the component type, as that will (almost?) always be redundant with the component name. For example, `stack build mypackage:mytestsuite`. + * `:compname` is a useful shortcut, saying "find the component in all of the local packages." This will result in an error if multiple packages have a component with the same name. To continue the above example, `stack build :mytestsuite`. + * Side note: the commonly requested `run` command is not available because it's a simple combination of `stack build :exename && stack exec exename` + +* *directory*, e.g. `stack build foo/bar`, will find all local packages that exist in the given directory hierarchy and then follow the same procedure as passing in package names as mentioned above. There's an important caveat here: if your directory name is parsed as one of the above target types, it will be treated as that. Explicitly starting your target with `./` can be a good way to avoid that, e.g. `stack build ./foo` + +Finally: if you provide no targets (e.g., running `stack build`), stack will +implicitly pass in all of your local packages. If you only want to target +packages in the current directory or deeper, you can pass in `.`, e.g. `stack +build .`. + +### Examples + +FIXME: what examples would be helpful? Need feedback on what's confusing above + +## Dependencies + +stack will always automatically build all dependencies necessary for its targets. + +## Other flags + +There are a number of other flags accepted by `stack build`. Instead of listing +all of them, please use `stack build --help`. Some particularly convenient ones +worth mentioning here since they compose well with the rest of the build system +as described: + +* `--file-watch` will rebuild your project every time a file changes +* `--exec "cmd [args]"` will run a command after a successful build + +To come back to the composable approach described above, consider this final +example (which uses the [wai repository](https://github.com/yesodweb/wai/): + +``` +stack build --file-watch --test --copy-bins --haddock wai-extra :warp warp:doctest --exec 'echo Yay, it worked!' +``` + +This command will: + +* Start stack up in file watch mode, waiting for files in your project to change. When first starting, and each time a file changes, it will do all of the following. +* Build the wai-extra package and its test suites +* Build the `warp` executable +* Build the warp package's doctest component (which, as you may guess, is a test site) +* Run all of the wai-extra package's test suite components and the doctest test suite component +* If all of that succeeds: + * Copy generated executables to the local bin path + * Run the command `echo Yay, it worked!` diff --git a/doc/dependency_visualization.md b/doc/dependency_visualization.md new file mode 100644 --- /dev/null +++ b/doc/dependency_visualization.md @@ -0,0 +1,43 @@ +# Dependency visualization + +You can use stack to visualize the dependencies between your packages and optionally also external dependencies. + +As an example, let's look at `wreq`: + +``` +$ stack dot | dot -Tpng -o wreq.png +``` +[![wreq](https://cloud.githubusercontent.com/assets/591567/8478591/ae10a418-20d2-11e5-8945-55246dcfac62.png)](https://cloud.githubusercontent.com/assets/591567/8478591/ae10a418-20d2-11e5-8945-55246dcfac62.png) + +Okay that is a little boring, let's also look at external dependencies: +``` +$ stack dot --external | dot -Tpng -o wreq.png +``` +[![wreq_ext](https://cloud.githubusercontent.com/assets/591567/8478621/d247247e-20d2-11e5-993d-79096e382abd.png)](https://cloud.githubusercontent.com/assets/591567/8478621/d247247e-20d2-11e5-993d-79096e382abd.png) + +Well that is certainly a lot. As a start we can exclude `base` and then depending on our needs we can either limit the depth: + +``` +$ stack dot --no-include-base --external --depth 1 | dot -Tpng -o wreq.png +``` +[![wreq_depth](https://cloud.githubusercontent.com/assets/591567/8484310/45b399a0-20f7-11e5-8068-031c2b352961.png)](https://cloud.githubusercontent.com/assets/591567/8484310/45b399a0-20f7-11e5-8068-031c2b352961.png) + +or prune packages explicitly: + +``` +$ stack dot --external --prune base,lens,wreq-examples,http-client,aeson,tls,http-client-tls,exceptions | dot -Tpng -o wreq_pruned.png +``` +[![wreq_pruned](https://cloud.githubusercontent.com/assets/591567/8478768/adbad280-20d3-11e5-9992-914dc24fe569.png)](https://cloud.githubusercontent.com/assets/591567/8478768/adbad280-20d3-11e5-9992-914dc24fe569.png) + +Keep in mind that you can also save the dot file: +``` +$ stack dot --external --depth 1 > wreq.dot +$ dot -Tpng -o wreq.png wreq.dot +``` + +and pass in options to `dot` or use another graph layout engine like `twopi`: + +``` +$ stack dot --external --prune base,lens,wreq-examples,http-client,aeson,tls,http-client-tls,exceptions | twopi -Groot=wreq -Goverlap=false -Tpng -o wreq_pruned.png +``` +[![wreq_pruned](https://cloud.githubusercontent.com/assets/591567/8495538/9fae1184-216e-11e5-9931-99e6147f8aed.png)](https://cloud.githubusercontent.com/assets/591567/8495538/9fae1184-216e-11e5-9931-99e6147f8aed.png) diff --git a/doc/docker_integration.md b/doc/docker_integration.md new file mode 100644 --- /dev/null +++ b/doc/docker_integration.md @@ -0,0 +1,475 @@ +Docker integration +=============================================================================== + +`stack` has support for automatically performing builds inside a Docker +container, using volume mounts and user ID switching to make it mostly seamless. +FP Complete provides images for use with stack that include GHC, tools, and +optionally have all of the Stackage LTS packages pre-installed in the global +package database. + +The primary purpose for using stack/docker this way is for teams to ensure all +developers are building in an exactly consistent environment without team +members needing to deal with Docker themselves. + +See the +[how stack can use Docker under the hood](https://www.fpcomplete.com/blog/2015/08/stack-docker) +blog post for more information about the motivation and implementation of stack's +Docker support. + +Usage +------------------------------------------------------------------------------- + +This section assumes that you already have Docker installed and working. If +not, see the [prerequisites](#prerequisites) section. If you run into any +trouble, see the [troubleshooting](#troubleshooting) section. + +### Enable in stack.yaml + +The most basic configuration is to add this to your project's `stack.yaml`: + + docker: + enable: true + +See [configuration](#configuration) for additional options. + +### Use stack as normal + +With Docker enabled, most stack sub-commands will automatically launch +themselves in an ephemeral Docker container (the container is deleted as soon as +the command completes). The project directory and `~/.stack` are volume-mounted +into the container, so any build artifacts are "permanent" (not deleted with the +container). + +The first time you run a command with a new image, you will be prompted to run +`stack docker pull` to pull the image first. This will pull a Docker +image with a tag that matches your resolver. Only LTS resolvers are supported +(we do not generate images for nightly snapshots). Not every LTS version is +guaranteed to have an image existing, and new LTS images tend to lag behind +the LTS snapshot being published on stackage.org. Be warned: these images are +rather large! + +Docker sub-commands +------------------------------------------------------------------------------- + +These `stack docker` sub-commands have Docker-specific functionality. Most other +`stack` commands will also use a Docker container under the surface if Docker is +enabled. + +### pull - Pull latest version of image + +`stack docker pull` pulls an image from the Docker registry for the first time, +or updates the image by pulling the latest version. + +### cleanup - Clean up old images and containers + +Docker images can take up quite a lot of disk space, and it's easy for them to +build up if you switch between projects or your projects update their images. +This sub-command will help to remove old images and containers. + +By default, `stack docker cleanup` will bring up an editor showing the images +and containers on your system, with any stack images that haven't been used +in the last seven days marked for removal. You can add or remove the `R` in +the left-most column to flag or unflag an image/container for removal. When +you save the file and quit the text editor, those images marked for removal +will be deleted from your system. If you wish to abort the cleanup, delete +all the lines from your editor. + +If you use Docker for purposes other than stack, you may have other images on +your system as well. These will also appear in a separate section, but they +will not be marked for removal by default. + +Run `stack docker cleanup --help` to see additional options to customize its +behaviour. + +### reset - Reset the Docker "sandbox" + +In order to preserve the contents of the in-container home directory between +runs, a special "sandbox" directory is volume-mounted into the container. `stack +docker reset` will reset that sandbox to its defaults. + +Note: `~/.stack` is separately volume-mounted, and is left alone during reset. + +Command-line options +------------------------------------------------------------------------------- + +The default Docker configuration can be overridden on the command-line. See +`stack --docker-help` for a list of all Docker options, and consult +[configuration](#configuration) section below for more information about +their meanings. These are global options, and apply to all commands (not just +`stack docker` sub-commands). + +Configuration +------------------------------------------------------------------------------- + +`stack.yaml` contains a `docker:` section with Docker settings. If this +section is omitted, Docker containers will not be used. These settings can +be included in project, user, or global configuration. + +Here is an annotated configuration file. The default values are shown unless +otherwise noted. + + docker: + + # Set to false to disable using Docker. In the project configuration, + # the presence of a `docker:` section implies docker is enabled unless + # `enable: false` is set. In user and global configuration, this is not + # the case. + enable: true + + # The name of the repository to pull the image from. See the "repositories" + # section of this document for more information about available repositories. + # If this includes a tag (e.g. "my/image:tag"), that tagged image will be + # used. Without a tag specified, the LTS version slug is added automatically. + # Either `repo` or `image` may be specified, but not both. + repo: "fpco/stack-build" + + # Exact Docker image name or ID. Overrides `repo`. Either `repo` or `image` + # may be specified, but not both. (default none) + image: "5c624ec1d63f" + + # Registry requires login. A login will be requested before attempting to + # pull. + registry-login: false + + # Username to log into the registry. (default none) + registry-username: "myuser" + + # Password to log into the registry. (default none) + registry-password: "SETME" + + # If true, the image will be pulled from the registry automatically, without + # needing to run `stack docker pull`. See the "security" section of this + # document for implications of enabling this. + auto-pull: false + + # If true, the container will be run "detached" (in the background). Refer + # to the Docker users guide for information about how to manage containers. + # This option would rarely make sense in the configuration file, but can be + # useful on the command-line. When true, implies `persistent`.` + detach: false + + # If true, the container will not be deleted after it terminates. Refer to + # the Docker users guide for information about how to manage containers. This + # option would rarely make sense in the configuration file, but can be + # useful on the command-line. `detach` implies `persistent`. + persistent: false + + # What to name the Docker container. Only useful with `detach` or + # `persistent` true. (default none) + container-name: "example-name" + + # Additional arguments to pass to `docker run`. (default none) + run-args: ["--net=bridge"] + + # Directories from the host to volume-mount into the container. If it + # contains a `:`, the part before the `:` is the directory on the host and + # the part after the `:` is where it should be mounted in the container. + # (default none, aside from the project and stack root directories which are + # always mounted) + mount: + - "/foo/bar" + - "/baz:/tmp/quux" + + # Environment variables to set in the container. Environment variables + # are not automatically inherited from the host, so if you need any specific + # variables, use the '--docker-env` command-line argument version of this to + # pass them in. (default none) + env: + - "FOO=BAR" + - "BAR=BAZ QUUX" + + # Location of database used to track image usage, which `stack docker cleanup` + # uses to determine which images should be kept. On shared systems, it may + # be useful to override this in the global configuration file so that + # all users share a single database. + database-path: "~/.stack/docker.db" + + # Location of a Docker container-compatible 'stack' executable with the + # matching version. This executable must be built on linux-x86_64 and + # statically linked. + # Valid values are: + # host: use the host's executable. This is the default when the host's + # executable is known to work (e.g., from official linux-x86_64 bindist) + # download: download a compatible executable matching the host's version. + # This is the default when the host's executable is not known to work + # image: use the 'stack' executable baked into the image. The version + # must match the host's version + # /path/to/stack: path on the host's local filesystem + stack-exe: host + + # If true (the default when using the local Docker Engine), run processes + # in the Docker container as the same UID/GID as the host. The ensures + # that files written by the container are owned by you on the host. + # When the Docker Engine is remote (accessed by tcp), defaults to false. + set-user: true + + # Require the version of the Docker client to be within the specified + # Cabal-style version range (e.g., ">= 1.6.0 && < 1.9.0") + require-docker-version: "any" + +Image Repositories +------------------------------------------------------------------------------- + +FP Complete provides the following public image repositories on Docker Hub: + +- [fpco/stack-build](https://registry.hub.docker.com/u/fpco/stack-build/) (the + default) - GHC (patched), tools (stack, cabal-install, happy, alex, etc.), and + system developer libraries required to build all Stackage packages. +- [fpco/stack-ghcjs-build](https://registry.hub.docker.com/u/fpco/stack-ghcjs-build/) - + Like `stack-build`, but adds GHCJS. +- [fpco/stack-full](https://registry.hub.docker.com/u/fpco/stack-full/) - + Includes all Stackage packages pre-installed in GHC's global package database. + These images are over 10 GB! +- [fpco/stack-ghcjs-full](https://registry.hub.docker.com/u/fpco/stack-ghcjs-full/) - + Like `stack-full`, but adds GHCJS. +- [fpco/stack-run](https://registry.hub.docker.com/u/fpco/stack-run/) - + Runtime environment for binaries built with Stackage. Includes system shared + libraries required by all Stackage packages. Does not necessarily include all + data required for every use (e.g. has texlive-binaries for HaTeX, but does not + include LaTeX fonts), as that would be prohibitively large. Based on + [phusion/baseimage](https://registry.hub.docker.com/u/phusion/baseimage/). + +FP Complete also builds custom variants of these images for their clients. + +These images can also be used directly with `docker run` and provide a complete +Haskell build environment. + +In addition, most Docker images that contain the basics for running GHC can be +used with Stack's Docker integration. For example, the +[official Haskell image repository](https://hub.docker.com/_/haskell/) works. +See [Custom images](#custom-images) for more details. + +Prerequisites +------------------------------------------------------------------------------- + +### Linux 64-bit + +Docker use requires machine (virtual or metal) running a Linux distribution +[that Docker supports](https://docs.docker.com/installation/#installation), with +a 64-bit kernel. If you do not already have one, we suggest Ubuntu 14.04 +("trusty") since this is what we test with. + +While Docker does support non-Linux operating systems through the `boot2docker` +VM, there are issues with host volume mounting that prevent stack from being +usable in this configuration. See +[#194](https://github.com/commercialhaskell/stack/issues/194) for details and +workarounds. + +### Docker + +Install the latest version of Docker by following the +[instructions for your operating system](http://docs.docker.com/installation/). + +The Docker client should be able to connect to the Docker daemon as a non-root +user. For example (from +[here](http://docs.docker.com/installation/ubuntulinux/#ubuntu-raring-1304-and-saucy-1310-64-bit)): + + # Add the connected user "${USER}" to the docker group. + # Change the user name to match your preferred user. + sudo gpasswd -a ${USER} docker + + # Restart the Docker daemon. + sudo service docker restart + +You will now need to log out and log in again for the group addition +to take effect. + +Note the above has security implications. See [security](#security) for more. + +Security +------------------------------------------------------------------------------- + +Having `docker` usable as a non-root user is always a security risk, and will +allow root access to your system. It is also possible to craft a `stack.yaml` +that will run arbitrary commands in an arbitrary docker container through that +vector, thus a `stack.yaml` could cause stack to run arbitrary commands as root. +While this is a risk, it is not really a greater risk than is posed by the +docker permissions in the first place (for example, if you ever run an unknown +shell script or executable, or ever compile an unknown Haskell package that uses +Template Haskell, you are at equal risk). Nevertheless, there are +[plans to close the stack.yaml loophole](https://github.com/commercialhaskell/stack/issues/260). + +One way to mitigate this risk is, instead of allowing `docker` to run as +non-root, replace `docker` with a wrapper script that uses `sudo` to run the +real Docker client as root. This way you will at least be prompted for your root +password. As [@gregwebs](https://github.com/gregwebs) pointed out, put this +script named `docker` in your PATH (and make sure you remove your user from the +`docker` group as well, if you added it earlier): + + #!/bin/bash -e + # The goal of this script is to maintain the security privileges of sudo + # Without having to constantly type "sudo" + exec sudo /usr/bin/docker "$@" + +Additional notes +------------------------------------------------------------------------------- + +### Volume-mounts and ephemeral containers + +Since filesystem changes outside of the volume-mounted project directory are not +persisted across runs, this means that if you `stack exec sudo apt-get install +some-ubuntu-package`, that package will be installed but then the container it's +installed in will disappear, thus causing it to have no effect. If you wish to +make this kind of change permanent, see later instructions for how to create a +[derivative Docker image](#derivative-image). + +Inside the container, your home directory is a special location that volume- +mounted from within your project directory's `.stack-work` in such a +way as that installed GHC/cabal packages are not shared between different +Stackage snapshots. In addition, `~/.stack` is volume-mounted from the host. + +### Network + +stack containers use the host's network stack within the container +by default, meaning a process running in the container can connect to +services running on the host, and a server process run within the container +can be accessed from the host without needing to explicitly publish its port. +To run the container with an isolated network, use `--docker-run-args` to pass +the `--net` argument to `docker-run`. For example: + + stack --docker-run-args='--net=bridge --publish=3000:3000' \ + exec some-server + +will run the container's network in "bridge" mode (which is Docker's default) +and publish port 3000. + +### Persistent container + +If you do want to do all your work, including editing, in the container, it +might be better to use a persistent container in which you can install Ubuntu +packages. You could get that by running something like `stack +--docker-container-name=NAME --docker-persist exec --plain bash`. This +means when the container exits, it won't be deleted. You can then restart it +using `docker start -a -i NAME`. It's also possible to detach from a container +while it continues running in the background using by pressing Ctrl-P Ctrl-Q, +and then reattach to it using `docker attach NAME`. + +Note that each time you run `stack --docker-persist`, a _new_ persistent +container is created (it will not automatically reuse the previous one). +See the [Docker user guide](https://docs.docker.com/userguide/) for more +information about managing Docker containers. + +### Derivative image + +Creating your own custom derivative image can be useful if you need to install +additional Ubuntu packages or make other changes to the operating system. Here +is an example (replace `stack-build:custom` if you prefer a different name for +your derived container, but it's best if the repo name matches what you're +deriving from, only with a different tag, to avoid recompilation): + + ;;; On host + $ sudo stack --docker-persist --docker-container-name=temp exec bash + + ;;; In container, make changes to OS + # apt-get install r-cran-numderiv + [...] + # exit + + ;;; On host again + $ docker commit temp stack-build:custom + $ docker rm temp + +Now you have a new Docker image named `stack-build:custom`. To use the new image, run +a command such as the following or update the corresponding values in your +`stack.yaml`: + + stack --docker-image=stack-build:custom + +Note, however, that any time a new image is used, you will have to re-do this +process. You could also use a Dockerfile to make this reusable. Consult the +[Docker user guide](https://docs.docker.com/userguide/) for more +on creating Docker images. + +### Custom images + +The easiest way to create your own custom image us by extending FP Complete's +images, but if you prefer to start from scratch, most images that include the +basics for building code with GHC will work. The image doesn't even, strictly +speaking, need to include GHC, but it does need to have libraries and tools that +GHC requires (e.g., libgmp, gcc, etc.). + +There are also a few ways to set up images that tightens the integration: + +* Create a user and group named `stack`, and create a `~/.stack` directory for + it. Any build plans and caches from it will be copied from the image by Stack, + meaning they don't need to be downloaded separately. +* Any packages in GHC's global package database will be available. This can be + used to add private libraries to the image, or the make available a set of + packages from an LTS release. + +Troubleshooting +------------------------------------------------------------------------------- + +### "No Space Left on Device", but 'df' shows plenty of disk space + +This is likely due to the storage driver Docker is using, in combination with +the large size and number of files in these images. Use `docker info|grep +'Storage Driver'` to determine the current storage driver. + +We recommend using either the `overlay` or `aufs` storage driver for stack, as +they are least likely to give you trouble. On Ubuntu, `aufs` is the default for +new installations, but older installations sometimes used `devicemapper`. + +The `devicemapper` storage driver's default configuration limits it to a 10 GB +file system, which the "full" images exceed. We have experienced other +instabilities with it as well on Ubuntu, and recommend against its use for this +purpose. + +The `btrfs` storage driver has problems running out of metadata space long +before running out of actual disk space, which requires rebalancing or adding +more metadata space. See +[CoreOS's btrfs troubleshooting page](https://coreos.com/docs/cluster-management/debugging/btrfs-troubleshooting/) +for details about how to do this. + +Pass the `-s ` argument to the Docker daemon to set the storage driver +(in `/etc/default/docker` on Ubuntu). See +[Docker daemon storage-driver option](https://docs.docker.com/reference/commandline/cli/#daemon-storage-driver-option) +for more details. + +You may also be running out of inodes on your filesystem. Use `df -i` to check +for this condition. Unfortunately, the number of inodes is set when creating +the filesystem, so fixing this requires reformatting and passing the `-N` +argument to mkfs.ext4. + +### Name resolution doesn't work from within container + +On Ubuntu 12.04, by default `NetworkManager` runs `dnsmasq` service, which sets +`127.0.0.1` as your DNS server. Since Docker containers cannot access this +dnsmasq, Docker falls back to using Google DNS (8.8.8.8/8.8.4.4). This causes +problems if you are forced to use internal DNS server. This can be fixed by +executing: + + sudo sed 's@dns=dnsmasq@#dns=dnsmasq@' -i \ + /etc/NetworkManager/NetworkManager.conf && \ + sudo service network-manager restart + +If you have already installed Docker, you must restart the daemon for this +change to take effect: + + sudo service docker restart + + +The above commands turn off `dnsmasq` usage in NetworkManager +configuration and restart network manager. They can be reversed by executing +`sudo sed 's@#dns=dnsmasq@dns=dnsmasq@' -i +/etc/NetworkManager/NetworkManager.conf && sudo service network-manager +restart`. These instructions are adapted from +[the Shipyard Project's QuickStart guide](https://github.com/shipyard/shipyard/wiki/QuickStart#127011-dns-server-problem-on-ubuntu). + + +### Cannot pull images from behind firewall that blocks TLS/SSL + +If you are behind a firewall that blocks TLS/SSL and pulling images from a +private Docker registry, you must edit the system configuration so that the +`--insecure-registry ` option is passed to the Docker daemon. +For example, on Ubuntu: + + echo 'DOCKER_OPTS="--insecure-registry registry.example.com"' \ + |sudo tee -a /etc/default/docker + sudo service docker restart + +This does require the private registry to be available over plaintext HTTP. + +See +[Docker daemon insecure registries documentation](https://docs.docker.com/reference/commandline/cli/#insecure-registries) +for details. diff --git a/doc/faq.md b/doc/faq.md new file mode 100644 --- /dev/null +++ b/doc/faq.md @@ -0,0 +1,333 @@ +# FAQ + +So that this doesn't become repetitive: for the reasons behind the answers +below, see the [Architecture](architecture.md) page. The goal of the answers here is to be as +helpful and concise as possible. + +## Where is stack installed and will it interfere with `ghc` (etc) I already have installed? + +Stack itself is installed in normal system locations based on the mechanism you used (see the [Install and upgrade](install_and_upgrade.md) page). Stack installs the Stackage libraries in `~/.stack` and any project libraries or extra dependencies in a `.stack-work` directory within each project's directory. None of this should affect any existing Haskell tools at all. + +## What is the relationship between stack and cabal? + +* Cabal-the-library is used by stack to build your Haskell code. +* cabal-install (the executable) is used by stack for its dependency solver functionality. +* A .cabal file is provided for each package, and defines all package-level metadata just like it does in the cabal-install world: modules, executables, test suites, etc. No change at all on this front. +* A stack.yaml file references 1 or more packages, and provides information on where dependencies come from. +* `stack build` currently initializes a stack.yaml from the existing .cabal file. Project initialization is something that is still being discussed and there may be more options here for new projects in the future (see issue [253](https://github.com/commercialhaskell/stack/issues/253)) + +## I need to use a different version of a package than what is provided by the LTS Haskell snapshot I'm using, what should I do? + +You can make tweaks to a snapshot by modifying the `extra-deps` configuration value in your `stack.yaml` file, e.g.: + +```yaml +resolver: lts-2.9 +packages: +- '.' +extra-deps: +- text-1.2.1.1 +``` + +## I need to use a package (or version of a package) that is not available on hackage, what should I do? + +Add it to the `packages` list in your project's `stack.yaml`, specifying the package's source code location relative to the directory where your `stack.yaml` file lives, e.g. + +```yaml +resolver: lts-2.10 +packages: +- '.' +- third-party/proprietary-dep +- github-version-of/conduit +- patched/diagrams +extra-deps: [] +``` + +The above example specifies that it should include the package at the root directory (`'.'`), that the `proprietary-dep` package is found in the project's `third-party` folder, that the `conduit` package is found in the project's `github-version-of` folder, and that the `diagrams` package is found in the project's `patched` folder. This autodetects changes and reinstalls the package. + +To install packages directly from a Git repository, use e.g.: + +```yaml +resolver: lts-2.10 +packages: +- location: + git: https://github.com/githubuser/reponame.git + commit: somecommitID +``` + +Note that the `- '.'` line has been omitted, so the package in the root directory will not be used. + +## What is the meaning of the arguments given to stack build, test, etc? + +Those are the targets of the build, and can have one of three formats: + +* A package name (e.g., `my-package`) will mean that the `my-package` package must be built +* A package identifier (e.g., `my-package-1.2.3`), which includes a specific version. This is useful for passing to `stack install` for getting a specific version from upstream +* A directory (e.g., `./my-package`) for including a local directory's package, including any packages in subdirectories + +## I need to modify an upstream package, how should I do it? + +Typically, you will want to get the source for the package and then add it to +your `packages` list in stack.yaml. (See the previous question.) +`stack unpack` is one approach for getting the source. +Another would be to add the upstream package as a submodule to your +project. + +## Am I required to use a Stackage snapshot to use stack? + +No, not at all. If you prefer dependency solving to curation, you can continue with that workflow. Instead of describing the details of how that works here, it's probably easiest to just say: run `stack init --solver` and look at the generated stack.yaml. + +## How do I use this with sandboxes? + +Explicit sandboxing on the part of the user is not required by stack. All +builds are automatically isolated into separate package databases without any +user interaction. This ensures that you won't accidentally corrupt your +installed packages with actions taken in other projects. + +## Can I run `cabal` commands inside `stack exec`? + +With a recent enough version of cabal-install (>= 1.22), you can. For older versions, due to [haskell/cabal#1800](https://github.com/haskell/cabal/issues/1800), this does not work. Note that even with recent versions, for some commands you may need this extra level of indirection: +``` +$ stack exec -- cabal exec -- cabal +``` + +However, virtually all `cabal` commands have an equivalent in stack, so this should not be necessary. In particular, `cabal` users may be accustomed to the `cabal run` command. In stack: +``` +$ stack build && stack exec +```` +Or, if you want to install the binaries in a shared location: +``` +$ stack install +$ +``` +assuming your `$PATH` has been set appropriately. + +## Using custom preprocessors + +If you have a custom preprocessor, for example, Ruby, you may have a +file like: + +***B.erb*** + +``` haskell +module B where + +<% (1..5).each do |i| %> +test<%= i %> :: Int +test<%= i %> = <%= i %> +<% end %> +``` + +To ensure that Stack picks up changes to this file for rebuilds, add +the following line to your .cabal file: + + extra-source-files: B.erb + +## I already have GHC installed, can I still use stack? + +Yes. stack will default to using whatever GHC is on your `PATH`. If that GHC is a +compatible version with the snapshot you're using, it will simply use it. +Otherwise, it will prompt you to run `stack setup`. Note that `stack setup` installs GHC into `~/.stack/programs/$platform/ghc-$version/` and not a global location. + +Note that GHC installation doesn't work for all OSes, so in some cases the +first option will need to install GHC yourself. + +## How does stack determine what GHC to use? + +It uses the first GHC that it finds on the `PATH`. If that GHC does not comply with the various requirements (version, architecture) that your project needs, it will prompt you to run `stack setup` to get it. `stack` is fully aware of all GHCs that it has installed itself. + +See [this issue](https://github.com/commercialhaskell/stack/issues/420) for a detailed discussion. + +## How do I upgrade to GHC 7.10.2 with stack? + +If you already have a prior version of GHC use `stack --resolver ghc-7.10 setup --reinstall`. If you don't have any GHC installed, you can skip the `--reinstall`. + +## How do I get extra build tools? + +stack will automatically install build tools required by your packages or their +dependencies, in particular alex and happy. + +__NOTE__: This works when using lts or nightly resolvers, not with ghc or custom resolvers. You can manually install build tools by running, e.g., `stack build alex happy`. + +## How does stack choose which snapshot to use when creating a new config file? + +It checks the two most recent LTS Haskell major versions and the most recent +Stackage Nightly for a snapshot that is compatible with all of the version +bounds in your .cabal file, favoring the most recent LTS. For more information, +see the snapshot auto-detection section in the architecture document. + +## I'd like to use my installed packages in a different directory. How do I tell stack where to find my packages? + +Set the `STACK_YAML` environment variable to point to the `stack.yaml` config +file for your project. Then you can run `stack exec`, `stack ghc`, etc., from +any directory and still use your packages. + +## My tests are failing. What should I do? + +Like all other targets, `stack test` runs test suites in parallel by default. This can cause problems with test suites that depend on global resources such as a database or binding to a fixed port number. A quick hack is to force stack to run all test suites in sequence, using `stack test --jobs=1`. For test suites to run in parallel developers should ensure that their test suites do not depend on global resources (e.g. by asking the OS for a random port to bind to) and where unavoidable, add a lock in order to serialize access to shared resources. + +## Can I get bash autocompletion? + +Yes, see the [shell-autocompletion documentation](shell_autocompletion.md) + +## How do I update my package index? + +Users of cabal are used to running `cabal update` regularly. You can do the +same with stack by running `stack update`. But generally, it's not necessary: +if the package index is missing, or if a snapshot refers to package/version +that isn't available, stack will automatically update and then try again. If +you run into a situation where stack doesn't automatically do the update for +you, please report it as a bug. + +## Isn't it dangerous to automatically update the index? Can't that corrupt build plans? + +No, stack is very explicit about which packages it's going to build for you. +There are three sources of information to tell it which packages to install: +the selected snapshot, the `extra-deps` configuration value, and your local +packages. The only way to get stack to change its build plan is to modify one +of those three. Updating the index will have no impact on stack's behavior. + +## I have a custom package index I'd like to use, how do I do so? + +You can configure this in your stack.yaml. See [YAML configuration](yaml_configuration.md). + +## How can I make sure my project builds against multiple ghc versions? + +You can create multiple yaml files for your project, +one for each build plan. For example, you might set up your project directory like so: + +``` +myproject/ + stack-7.8.yaml + stack-7.10.yaml + stack.yaml --> symlink to stack-7.8.yaml + myproject.cabal + src/ + ... +``` + +When you run `stack build`, you can set the +`STACK_YAML` environment variable to indicate which build plan to use. + +``` +$ stack build # builds using the default stack.yaml +$ STACK_YAML=stack-7.10.yaml stack build # builds using the given yaml file +``` + +## I heard you can use this with Docker? + +Yes, stack supports using Docker with images that contain preinstalled Stackage +packages and the tools. See [Docker integration](docker_integration.md) for details. + +## How do I use this with Travis CI? + +See the [Travis section in the GUIDE](GUIDE.md#travis-with-caching) + +## What is licensing restrictions on Windows? + +Currently on Windows GHC produces binaries linked statically with [GNU Multiple Precision Arithmetic Library](https://gmplib.org/) (GMP), which is used by [integer-gmp](https://hackage.haskell.org/package/integer-gmp) library to provide big integer implementation for Haskell. Contrary to the majority of Haskell code licensed under permissive BSD3 license, GMP library is licensed under LGPL, which means resulting binaries [have to be provided with source code or object files](http://www.gnu.org/licenses/gpl-faq.html#LGPLStaticVsDynamic). That may or may not be acceptable for your situation. Current workaround is to use GHC built with alternative big integer implementation called integer-simple, which is free from LGPL limitations as it's pure Haskell and does not use GMP. Unfortunately it has yet to be available out of the box with stack. See [issue #399](https://github.com/commercialhaskell/stack/issues/399) for the ongoing effort and information on workarounds. + +## How to get a working executable on Windows? + +When executing a binary after building with `stack build` (e.g. for target "foo"), the command `foo.exe` might complain about missing runtime libraries (whereas `stack exec foo` works). + +Windows is not able to find the necessary C++ libraries from the standard prompt because they're not in the PATH environment variable. `stack exec` works because it's modifying PATH to include extra things. + +Those libraries are shipped with GHC (and, theoretically in some cases, MSYS). The easiest way to find them is `stack exec which`. E.g. + + >stack exec which libstdc++-6.dll + /c/Users/Michael/AppData/Local/Programs/stack/i386-windows/ghc-7.8.4/mingw/bin/libstdc++-6.dll + +A quick workaround is adding this path to the PATH environment variable or copying the files somewhere Windows finds them (cf. https://msdn.microsoft.com/de-de/library/7d83bc18.aspx). + +Cf. issue [#425](https://github.com/commercialhaskell/stack/issues/425). + +## Can I change stack's default temporary directory? + +Stack makes use of a temporary directory for some commands (/tmp by default on linux). If there is not enough free space in this directory, stack may fail (see issue [#429](https://github.com/commercialhaskell/stack/issues/429) ). For instance `stack setup` with a GHC installation requires roughly 1GB free. + +A custom temporary directory can be forced: +* on Linux by setting the environment variable TMPDIR (eg `$ TMPDIR=path-to-tmp stack setup`) +* on Windows by setting one of the environment variable (given in priority order), TMP, TEMP, USERPROFILE + +If you use Stack with Nix integration, be aware that Nix _also_ uses that TMPDIR +variable, and if it is not set Nix sets it to some subdirectory of `/run`, which +on most Linuxes is a Ramdir. Nix will run the builds in TMPDIR, therefore if you +don't have enough RAM you will get errors about disk space. If this happens to +you, please _manually_ set TMPDIR before launching Stack to some directory on the +disk. + +## stack sometimes rebuilds based on flag changes when I wouldn't expect it to. How come? + +stack tries to give you reproducibility whenever possible. In some cases, this means that you get a recompile when one may not seem necessary. The most common example is running something like this in a multi-package project: + + stack build --ghc-options -O0 && stack build --ghc-options -O0 one-of-the-packages + +This may end up recompiling local dependencies of `one-of-the-packages` without optimizations on. Whether stack should or shouldn't do this depends on the needs of the user at the time, and unfortunately we can't make a solution that will make everyone happy in all cases. If you're curious for details, there's [a long discussion about it](https://github.com/commercialhaskell/stack/issues/382) on the issue tracker. + +## stack setup on a windows system only tells me to add certain paths to the PATH variable instead of doing it + +If you are using a powershell session, it is easy to automate even that step: + + $env:Path = ( stack setup | %{ $_ -replace '[^ ]+ ', ''} ), $env:Path -join ";" + +## How do I reset / remove Stack (such as to to do a completely fresh build)? + +The first thing to remove is project-specific `.stack-work` directory within the project's directory. Next, remove `~/.stack` directory overall. You may have errors if you remove the latter but leave the former. Removing Stack itself will relate to how it was installed, and if you used GHC installed outside of Stack, that would need to be removed separately. + +## How does stack handle parallel builds? What exactly does it run in parallel? + +See [issue #644](https://github.com/commercialhaskell/stack/issues/644) for more details. + +## I get strange `ld` errors about recompiling with "-fPIC" + +Some users (myself included!) have come across a linker errors (example below) that seem to be dependent on the local environment, i.e. the package may compile on a different machine. There is no known workaround (if you come across one please include details), however the issue has been reported to be [non-deterministic](https://github.com/commercialhaskell/stack/issues/614) +in some cases. I've had success using the docker functionality to build the project on a machine that would not compile it otherwise. + +``` +tmp-0.1.0.0: build +Building tmp-0.1.0.0... +Preprocessing executable 'tmp' for tmp-0.1.0.0... +Linking dist-stack/x86_64-linux/Cabal-1.22.2.0/build/tmp/tmp ... +/usr/bin/ld: dist-stack/x86_64-linux/Cabal-1.22.2.0/build/tmp/tmp-tmp/Main.o: relocation R_X86_64_32S against `stg_bh_upd_frame_info' can not be used when making a shared object; recompile with -fPIC +dist-stack/x86_64-linux/Cabal-1.22.2.0/build/tmp/tmp-tmp/Main.o: error adding symbols: Bad value +collect2: error: ld returned 1 exit status + +-- While building package tmp-0.1.0.0 using: + /home/philip/.stack/programs/x86_64-linux/ghc-7.10.1/bin/runghc-7.10.1 -package=Cabal-1.22.2.0 -clear-package-db -global-package-db /home/philip/tmp/Setup.hs --builddir=dist-stack/x86_64-linux/Cabal-1.22.2.0/ build + Process exited with code: ExitFailure 1 +``` + +## Where does the output from `--ghc-options=-ddump-splices` (and other `-ddump*` options) go? + +These are written to `*.dump-*` files inside the package's `.stack-work` directory. + +## Why is DYLD_LIBRARY_PATH ignored? + +If you are on Mac OS X 10.11 ("El Capitan") or later, there is an +[upstream GHC issue](https://ghc.haskell.org/trac/ghc/ticket/11617) +which +[prevents the `DYLD_LIBRARY_PATH` environment variable from being passed to GHC](https://github.com/commercialhaskell/stack/issues/1161) +when System Integrity Protection (a.k.a. "rootless") is enabled. There are two +known workarounds: + + 1. Known to work in all cases: [disable System Integrity Protection](http://osxdaily.com/2015/10/05/disable-rootless-system-integrity-protection-mac-os-x/). **WARNING: Disabling SIP will severely reduce the security of your system, so only do this if absolutely necessary!** + 2. Experimental: [modify GHC's shell script wrappers to use a shell outside the protected directories](https://github.com/commercialhaskell/stack/issues/1161#issuecomment-186690904). + +## Why do I get a `/usr/bin/ar: permission denied` error? + +If you are on OS X 10.11 ("El Capitan") or +later, GHC 7.8.4 is +[incompatible with System Integrity Protection (a.k.a. "rootless")](https://github.com/commercialhaskell/stack/issues/563). +GHC 7.10.2 includes a fix, so this only affects users of GHC 7.8.4. If you +cannot upgrade to GHC 7.10.2, you can work around it by +[disabling System Integrity Protection](http://osxdaily.com/2015/10/05/disable-rootless-system-integrity-protection-mac-os-x/). **WARNING: Disabling SIP will severely reduce the security of your system, so only do this if absolutely necessary!** + +## Why is the `--` argument separator ignored in Windows PowerShell + +Some versions of Windows PowerShell +[don't pass the `--` to programs](https://github.com/commercialhaskell/stack/issues/813). +The workaround is to quote the `"--"`, e.g.: + + stack exec "--" cabal --version + +This is known to be a problem on Windows 7, but seems to be fixed on Windows 10. diff --git a/doc/ghcjs.md b/doc/ghcjs.md new file mode 100644 --- /dev/null +++ b/doc/ghcjs.md @@ -0,0 +1,88 @@ +# GHCJS + +To use GHCJS with stack `>= 0.1.8`, place a GHCJS version in the [`compiler`](yaml_configuration.md#compiler) field of `stack.yaml`. After this, all stack commands should work with GHCJS, except for `ide`. In particular: + +* `stack setup` will install GHCJS from source and boot it, which takes a long time. +* `stack build` will compile your code to JavaScript. In particular, the generated code for an executable ends up in `$(stack path --local-install-root)/bin/EXECUTABLE.jsexe/all.js` (bash syntax, where `EXECUTABLE` is the name of your executable). + +You can also build existing stack projects which target GHC, and instead build them with GHCJS. For example: `stack build --compiler ghcjs-0.1.0.20150924_ghc-7.10.2` + +Sidenote: If you receive a message like `The program 'ghcjs' version >=0.1 is +required but the version of .../ghcjs could not be determined.`, then you may +need to install a different version of `node`. See +[stack issue #1496](https://github.com/commercialhaskell/stack/issues/1496). + +## Example Configurations + +### GHCJS `master` (a.k.a. improved base) + +To use the master branch, a.k.a improved base, add the following to your `stack.yaml`: + +GHCJS compiled with GHC 7.10.3 LTS-5.12 (stack.yaml upgraded from stock GHCJS) +```yaml +compiler: ghcjs-0.2.0.20160414_ghc-7.10.3 +compiler-check: match-exact +setup-info: + ghcjs: + source: + ghcjs-0.2.0.20160414_ghc-7.10.3: + url: https://s3.amazonaws.com/ghcjs/ghcjs-0.2.0.20160414_ghc-7.10.3.tar.gz + sha1: 6d6f307503be9e94e0c96ef1308c7cf224d06be3 +``` + +GHCJS compiled with GHC 7.10.2 LTS-3.6 (stack.yaml that comes with GHCJS) +```yaml +compiler: ghcjs-0.2.0.20160414_ghc-7.10.2 +compiler-check: match-exact +setup-info: + ghcjs: + source: + ghcjs-0.2.0.20160414_ghc-7.10.2: + url: https://s3.amazonaws.com/ghcjs/ghcjs-0.2.0.20160414_ghc-7.10.2.tar.gz + sha1: f0a7243e781e27ebfe601eebaf5c57422007c142 +``` + +### GHCJS (old base) + +You can use this resolver for GHCJS (old base) in your `stack.yaml`: + +```yaml +compiler: ghcjs-0.1.0.20150924_ghc-7.10.2 +compiler-check: match-exact +``` + +### Custom installed GHCJS (development branch) + +In order to use a GHCJS installed on your path, just add the following to your `stack.yaml`: + +```yaml +compiler: ghcjs-0.2.0_ghc-7.10.2 +``` + +(Or, `ghcjs-0.1.0_ghc-7.10.2` if you are working with an older version) + +## Project with both client and server + +For projects with both a server and client, the recommended project organization is to put one or both of your `stack.yaml` files in sub-directories. This way, you can use the current working directory to specify whether you're working on the client or server. This will also allow more straightforward editor tooling, once projects like `ghc-mod` and `haskell-ide-engine` support GHCJS. + +For example, here's what a script for building both client and server looks like: + +```bash +#!/bin/bash + +# Build the client +stack build --stack-yaml=client/stack.yaml + +# Copy over the javascript +rm -f server/static/all.js +cp $(stack path --stack-yaml=client/stack.yaml --local-install-root)/bin/client.jsexe/all.js server/static/all.js + +# Build the server +stack build --stack-yaml=server/stack.yaml +``` + +You can also put both the yaml files in the same directory, and have e.g. `ghcjs-stack.yaml`, but this won't work well with editor integrations. + +## Using stack without a snapshot + +If you don't want to use a snapshot, instead place the ghcjs version in the `resolver` field of your `stack.yaml`. This is also necessary when using stack `< 0.1.8`. diff --git a/doc/install_and_upgrade.md b/doc/install_and_upgrade.md new file mode 100644 --- /dev/null +++ b/doc/install_and_upgrade.md @@ -0,0 +1,302 @@ +# Install/upgrade + +Distribution packages are available for [Ubuntu](#ubuntu), [Debian](#debian), +[CentOS / Red Hat / Amazon Linux](#centos), [Fedora](#fedora), +[Arch Linux](#arch-linux) and unofficially [FreeBSD](#freebsd). +Binaries for other operating systems are listed below, and available on +[the Github releases page](https://github.com/fpco/stack/releases). For the +future, we are open to supporting more OSes (to request one, please +[submit an issue](https://github.com/commercialhaskell/stack/issues/new)). + +Binary packages are signed with this [signing key](SIGNING_KEY.md). + +If you are writing a script that needs to download the latest binary, you can +find links that always point to the latest bindists +[here](https://www.stackage.org/stack). + +## Windows + +*Note*: Due to specific Windows limitations, + [some temporary workarounds](https://www.fpcomplete.com/blog/2015/08/stack-ghc-windows) + may be required. It is strongly advised to set your `STACK_ROOT` environment + variable similarly to your root (e.g., `set STACK_ROOT=c:\stack_root`) *before* + running `stack`. + +*Note:* while generally 32-bit GHC is better tested on Windows, there are +reports that recent versions of Windows only work with the 64-bit version of +Stack (see +[issue #393](https://github.com/commercialhaskell/stack/issues/393)). + +### Installer + +We recommend installing to the default location with these installers, as that +will make `stack install` and `stack upgrade` work correctly out of the box. + + * [Windows 32-bit Installer](https://www.stackage.org/stack/windows-i386-installer) + * [Windows 64-bit Installer](https://www.stackage.org/stack/windows-x86_64-installer) + +### Manual download + +* Download the latest release: + + * [Windows 32-bit](https://www.stackage.org/stack/windows-i386) + * [Windows 64-bit](https://www.stackage.org/stack/windows-x86_64) + +* Unpack the archive and place `stack.exe` somewhere on your `%PATH%` (see + [Path section below](#path)) and you can then run `stack` on the command line. + +* Now you can run `stack` from the terminal. + +NOTE: These executables have been built and tested on a Windows 7, 8.1, and 10 +64-bit machines. They should run on older Windows installs as well, but have not +been tested. If you do test, please edit and update this page to indicate as +such. + +## Mac OS X + +### Using Homebrew + +If you have a popular [brew](http://brew.sh/) tool installed, you can just do: + +``` +brew install haskell-stack +``` + +* The Homebrew formula and bottles lag slightly behind new Stack releases, +but tend to be updated within a day or two. +* At later stage, running `stack setup` might fail with `configure: error: cannot run C compiled programs.` in which case you should run `xcode-select --install`. +* Normally, Homebrew will install from a pre-built binary (aka "pour from a +bottle"), but if `brew` starts trying to build everything from source (which +will take hours), see +[their FAQ on the topic](https://github.com/Homebrew/homebrew/blob/master/share/doc/homebrew/FAQ.md#why-do-you-compile-everything). + +### Manual download + +* Download the latest release: + * [Mac OS X 64-bit](https://www.stackage.org/stack/osx-x86_64) +* Extract the archive and place `stack` somewhere on your `$PATH` (see + [Path section below](#path)) +* Now you can run `stack` from the terminal. + +We generally test on the current version of Mac OS X, but stack is known to work on +Yosemite and Mavericks as well, and may also work on older versions (YMMV). + +### Notes + +If you are on OS X 10.11 ("El Capitan") and encounter either of these +problems, see the linked FAQ entries: + + * [GHC 7.8.4 fails with `/usr/bin/ar: permission denied`](faq.md#usr-bin-ar-permission-denied) + * [DYLD_LIBRARY_PATH is ignored](faq.md#dyld-library-path-ignored) + + +## Ubuntu + +*note*: for 32-bit, use the [generic Linux option](#linux) + + 1. Get the FP Complete key: + + sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 575159689BEFB442 + + 2. Add the appropriate source repository (if not sure, run ``lsb_release -a`` to find out your Ubuntu version): + + * Ubuntu 16.04 (amd64): + + echo 'deb http://download.fpcomplete.com/ubuntu xenial main'|sudo tee /etc/apt/sources.list.d/fpco.list + + * Ubuntu 15.10 (amd64): + + echo 'deb http://download.fpcomplete.com/ubuntu wily main'|sudo tee /etc/apt/sources.list.d/fpco.list + + * Ubuntu 14.04 (amd64) + + echo 'deb http://download.fpcomplete.com/ubuntu trusty main'|sudo tee /etc/apt/sources.list.d/fpco.list + + * Ubuntu 12.04 (amd64) + + echo 'deb http://download.fpcomplete.com/ubuntu precise main'|sudo tee /etc/apt/sources.list.d/fpco.list + + 3. Update apt and install + + sudo apt-get update && sudo apt-get install stack -y + +## Debian + +*note*: for 32-bit, use the [generic Linux option](#linux) + + 1. Get the FP Complete key: + + sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 575159689BEFB442 + + 2. Add the appropriate source repository: + + * Debian 8 (amd64): + + echo 'deb http://download.fpcomplete.com/debian jessie main'|sudo tee /etc/apt/sources.list.d/fpco.list + + * Debian 7 (amd64) + + echo 'deb http://download.fpcomplete.com/debian wheezy main'|sudo tee /etc/apt/sources.list.d/fpco.list + + For unstable Debian distributions, the package from the most recent stable + release will usually work. If it doesn't, please + [report it](https://github.com/commercialhaskell/stack/issues/new). + + 3. Update apt and install + + sudo apt-get update && sudo apt-get install stack -y + +## CentOS / Red Hat / Amazon Linux + +*note*: for 32-bit, use the [generic Linux option](#linux) + + 1. Add the appropriate source repository: + + * CentOS 7 / RHEL 7 (x86_64) + + curl -sSL https://s3.amazonaws.com/download.fpcomplete.com/centos/7/fpco.repo | sudo tee /etc/yum.repos.d/fpco.repo + + * CentOS 6 / RHEL 6 (x86_64) + + curl -sSL https://s3.amazonaws.com/download.fpcomplete.com/centos/6/fpco.repo | sudo tee /etc/yum.repos.d/fpco.repo + + 2. Install: + + sudo yum -y install stack + +## Fedora + +*Note*: for 32-bit, you can use this + [Fedora Copr repo](https://copr.fedoraproject.org/coprs/petersen/stack/) (not + managed by the Stack release team, so not guaranteed to have the very latest + version) which can be enabled with: `sudo dnf copr enable petersen/stack` + + 1. Add the appropriate source repository: + + * Fedora 23 (x86_64) + + curl -sSL https://s3.amazonaws.com/download.fpcomplete.com/fedora/23/fpco.repo | sudo tee /etc/yum.repos.d/fpco.repo + + * Fedora 22 (x86_64) + + curl -sSL https://s3.amazonaws.com/download.fpcomplete.com/fedora/22/fpco.repo | sudo tee /etc/yum.repos.d/fpco.repo + + 2. Install: + + sudo dnf -y install stack + +## openSUSE / SUSE Linux Enterprise + +*Note:* openSUSE's and SLE's `stack` package isn't managed by the Stack release +team, and since it is based on the version in Stackage LTS, and may lag new +releases by ten days or more. + + 1. Add the appropriate OBS repository: + + * openSUSE Tumbleweed + + all needed is in distribution + + * openSUSE Leap + + sudo zypper ar http://download.opensuse.org/repositories/devel:/languages:/haskell/openSUSE_Leap_42.1/devel:languages:haskell.repo + + * SUSE Linux Enterprise 12 + + sudo zypper ar http://download.opensuse.org/repositories/devel:/languages:/haskell/SLE_12/devel:languages:haskell.repo + + 2. Install: + + sudo zypper in stack + +## Arch Linux + +*Note:* `stack` package in the [community] repository isn't managed by the +Stack release team. Depending on the maintainer's availability, it can lag +new releases by some days. + + - [stack](https://www.archlinux.org/packages/community/x86_64/stack/) _latest stable version_ + - [haskell-stack-git](https://aur.archlinux.org/packages/haskell-stack-git/) _git version_ + +In order to install stack from Hackage or from source, you will need the [libtinfo-5](https://aur.archlinux.org/packages/libtinfo-5/) Arch Linux package installed. If this package is not installed, stack will not be able to install GHC. + +If you use the [ArchHaskell repository](https://wiki.archlinux.org/index.php/ArchHaskell), you can also get the `haskell-stack-tool` package from there. + +## NixOS + +Users who follow the `nixos-unstable` channel or the Nixpkgs `master` branch can install the latest `stack` release into their profile by running: + + nix-env -f "" -iA haskellPackages.stack + +Alternatively, the package can be built from source as follows. + + 1. Clone the git repo: + + git clone https://github.com/commercialhaskell/stack.git + + 2. Create a `shell.nix` file: + + cabal2nix --shell ./. --no-check --no-haddock > shell.nix + + Note that the tests fail on NixOS, so disable them with `--no-check`. Also, haddock currently doesn't work for stack, so `--no-haddock` disables it. + + 3. Install stack to your user profile: + + nix-env -i -f shell.nix + +For more information on using Stack together with Nix, please see [the NixOS +manual section on +Stack](http://nixos.org/nixpkgs/manual/#using-stack-together-with-nix). + +## Linux + +(64-bit and 32-bit options available) + +* Download the latest release: + + * [Linux 64-bit, standard](https://www.stackage.org/stack/linux-x86_64) + * [Linux 32-bit, standard](https://www.stackage.org/stack/linux-i386) + + If you are on an older distribution that only includes libgmp4 (libgmp.so.3), such as CentOS/RHEL/Amazon Linux 6.x, use one of these instead: + + * [Linux 64-bit, libgmp4](https://www.stackage.org/stack/linux-x86_64-gmp4) + * [Linux 32-bit, libgmp4](https://www.stackage.org/stack/linux-i386-gmp4) + +* Extract the archive and place `stack` somewhere on your `$PATH` (see [Path section below](#path)) + +* Now you can run `stack` from the terminal. + +Tested on Fedora 20: make sure to install the following packages `sudo yum install perl make automake gcc gmp-devel`. +For Gentoo users, make sure to have the `ncurses` package with `USE=tinfo` (without it, stack will not be able to install GHC). + +## FreeBSD + +An unofficial package repository for FreeBSD 10 (amd64 only) and install +instructions are available at [http://stack-pkg.applicative.tech](http://stack-pkg.applicative.tech/). The +repository is not official and as such might lag behind new releases. See [issue #1253](https://github.com/commercialhaskell/stack/issues/1253) +for progress on official FreeBSD binaries. + +## Path + +You can install stack by copying it anywhere on your PATH environment variable. We recommend installing in the same directory where stack itself will install executables (that way stack is able to upgrade itself!). On Windows, that directory is `%APPDATA%\local\bin`, e.g. "c:\Users\Michael\AppData\Roaming\local\bin". For other systems, use `$HOME/.local/bin`. + +If you don't have that directory in your PATH, you may need to update your PATH (such as by editing .bashrc). + +If you're curious about the choice of these paths, see [issue #153](https://github.com/commercialhaskell/stack/issues/153) + +## Shell auto-completion + +To get tab-completion of commands on bash, just run the following (or add it to +`.bashrc`): + + eval "$(stack --bash-completion-script stack)" + +For more information and other shells, see [the shell auto-completion page](shell_autocompletion.md) + +## Upgrade + +There are essentially three different approaches to upgrade: + +* If you're using a package manager (e.g., the Ubuntu debs listed above) and are happy with sticking with the officially released binaries, simply follow your normal package manager strategies for upgrading (e.g. `apt-get update && apt-get upgrade`). +* If you're not using a package manager but want to stick with the official binaries (such as on Windows or Mac), you'll need to manually follow the steps above to download the newest binaries from the release page and replace the old binary. +* The `stack` tool itself ships with an `upgrade` command, which will build `stack` from source and install it to the default install path (see the previous section). You can use `stack upgrade` to get the latest official release, and `stack upgrade --git` to install from Git and live on the bleeding edge. If you follow this, make sure that this directory is on your `PATH` and takes precedence over the system installed `stack`. For more information, see [this discussion](https://github.com/commercialhaskell/stack/issues/237#issuecomment-126793301). diff --git a/doc/nix_integration.md b/doc/nix_integration.md new file mode 100644 --- /dev/null +++ b/doc/nix_integration.md @@ -0,0 +1,231 @@ +# Nix integration + +(since 0.1.10.0) + +When using the Nix integration, Haskell dependencies are handled as usual: they +are downloaded from Stackage and build locally by Stack. Nix is used by Stack to +provide the _non-Haskell_ dependencies needed by these Haskell packages. + +`stack` can automatically create a build environment (the equivalent +of a "container" in Docker parlance) using `nix-shell`, provided Nix +is already installed on your system. To do so, please visit the +[Nix download page](http://nixos.org/nix/download.html). + +There are two ways to create a build environment: + +- providing a list of packages (by "attribute name") from + [Nixpkgs](http://nixos.org/nixos/packages.html), or +- providing a custom `shell.nix` file containing a Nix expression that + determines a *derivation*, i.e. a specification of what resources + are available inside the shell. + +The second requires writing code in Nix's custom language. So use this +option only if you already know Nix and have special requirements, +such as using custom Nix packages that override the standard ones or +using system libraries with special requirements. + +### Additions to your `stack.yaml` + +Add a section to your `stack.yaml` as follows: +```yaml +nix: + enable: true + packages: [glpk, pcre] +``` + +This will instruct `stack` to build inside a local build environment +that will have the `glpk` and `pcre` libraries installed and +available. Further, the build environment will implicitly also include +a version of GHC matching the configured resolver. Enabling Nix +support means packages will always be built using a GHC available +inside the shell, rather than your globally installed one if any. + +Note that in this mode `stack` can use only those resolvers that have +already been mirrored into the Nix package repository. The +[Nixpkgs master branch](https://github.com/NixOS/nixpkgs/tree/master/pkgs/development/haskell-modules) +usually picks up new resolvers such as Stackage nightlies and LTS +versions within two or three days. Then it takes another two or three +days before those updates arrive in the `unstable` channel. Release +channels, like `nixos-15.09`, receive those updates only +occasionally -- say, every two or three months --, so you should not +expect them to have the latest resolvers available. Fresh Nix installs +use a release version by default. + +To know for sure whether a given resolver as available on your system, +you can use the command + +```sh +$ nix-env -f "" -qaP -A haskell.packages.lts-3_13.ghc +haskell.packages.lts-3_13.ghc ghc-7.10.2 +``` + +to check whether it's available. If Nix doesn't know that resolver +yet, then you'll see the following error message instead: + +```sh +$ nix-env -f "" -qaP -A haskell.packages.lts-3_99.ghc +error: attribute ‘lts-3_99’ in selection path ‘haskell.packages.lts-3_99.ghc’ not found +``` + +You can list all known Haskell package sets in Nix with the following: + +```sh +$ nix-instantiate --eval -E "with import {}; lib.attrNames haskell.packages" +``` + +Alternatively, install `nix-repl`, a convenient tool to explore +nixpkgs: + +```sh +$ nix-env -i nix-repl +$ nix-repl +``` + +In the REPL, load nixpkgs and get the same information through +autocomplete: + +```sh +nix-repl> :l +nix-repl> haskell.packages.lts- +``` + +You can type and evaluate any nix expression in the nix-repl, such as +the one we gave to `nix-instantiate` earlier. + +**Note:** currently, stack only discovers dynamic and static libraries +in the `lib/` folder of any nix package, and likewise header files in +the `include/` folder. If you're dealing with a package that doesn't +follow this standard layout, you'll have to deal with that using +a custom shell file (see below). + +### Use stack as normal + +With Nix enabled, `stack build` and `stack exec` will automatically +launch themselves in a local build environment (using `nix-shell` +behind the scenes). + +If `enable:` is omitted or set to `false`, you can still build in a nix-shell by +passing the `--nix` flag to stack, for instance `stack --nix build`. Passing +any `--nix*` option to the command line will do the same. + +**Known limitation on OS X:** currently, `stack --nix ghci` fails on +OS X, due to a bug in GHCi when working with external shared +libraries. + +### The Nix shell + +By default, stack will run the build in a pure Nix build environment +(or *shell*), which means the build should fail if you haven't +specified all the dependencies in the `packages:` section of the +`stack.yaml` file, even if these dependencies are installed elsewhere +on your system. This behaviour enforces a complete description of the +build environment to facilitate reproducibility. To override this +behaviour, add `pure: false` to your `stack.yaml` or pass the +`--no-nix-pure` option to the command line. + +**Note:** On OS X shells are non-pure by default currently. This is +due soon to be resolved locale issues. So on OS X you'll need to be +a bit more careful to check that you really have listed all +dependencies. + +### Package sources + +By default, `nix-shell` will look for the nixpkgs package set located +by your `NIX_PATH` environment variable. + +You can override this by passing +`--nix-path="nixpkgs=/my/own/nixpkgs/clone"` to ask Nix to use your +own local checkout of the nixpkgs repository. You could in this way +use a bleeding edge nixpkgs, cloned from the +[nixpkgs](http://www.github.com/NixOS/nixpkgs) `master` branch, or +edit the nix descriptions of some packages. Setting + +```yml +nix: + path: [nixpkgs=/my/own/nixpkgs/clone] +``` + +in your `stack.yaml` will do the same. + +## Command-line options + +The configuration present in your `stack.yaml` can be overridden on the +command-line. See `stack --nix-help` for a list of all Nix options. + +## Configuration + +`stack.yaml` contains a `nix:` section with Nix settings. +Without this section, Nix will not be used. + +Here is a commented configuration file, showing the default values: + +```yaml +nix: + + # false by default. Must be present and set to `true` to enable Nix. + # You can set set it in your `$HOME/.stack/config.yaml` to enable + # Nix for all your projects without having to repeat it + # enable: true + + # true by default. Tells Nix whether to run in a pure shell or not. + pure: true + + # Empty by default. The list of packages you want to be + # available in the nix-shell at build time (with `stack + # build`) and run time (with `stack exec`). + packages: [] + + # Unset by default. You cannot set this option if `packages:` + # is already present and not empty. + shell-file: shell.nix + + # A list of strings, empty by default. Additional options that + # will be passed verbatim to the `nix-shell` command. + nix-shell-options: [] + + # A list of strings, empty by default, such as + # `[nixpkgs=/my/local/nixpkgs/clone]` that will be used to override + # NIX_PATH. + path: [] +``` + +## Using a custom shell.nix file + +Nix is also a programming language, and as specified +[here](#nix-integration) if you know it you can provide to the shell +a fully customized derivation as an environment to use. Here is the +equivalent of the configuration used in +[this section](#additions-to-your-stackyaml), but with an explicit +`shell.nix` file (make sure you're using a nixpkgs version later than +2015-03-05): + +```nix +with (import {}); + +haskell.lib.buildStackProject { + name = "myEnv"; + buildInputs = [ glpk pcre ]; +} +``` + +Defining manually a `shell.nix` file gives you the possibility to +override some Nix derivations ("packages"), for instance to change +some build options of the libraries you use, or to set additional +environment variables. See the [Nix manual][nix-manual-exprs] for +more. The `buildStackProject` utility function is documented in the +[Nixpkgs manual][nixpkgs-manual-haskell]. + +And now for the `stack.yaml` file: + +```yaml +nix: + enable: true + shell-file: shell.nix +``` + +The `stack build` command will behave exactly the same as above. Note +that specifying both `packages:` and a `shell-file:` results in an +error. (Comment one out before adding the other.) + +[nix-manual-exprs]: http://nixos.org/nix/manual/#chap-writing-nix-expressions +[nixpkgs-manual-haskell]: https://nixos.org/nixpkgs/manual/#users-guide-to-the-haskell-infrastructure diff --git a/doc/nonstandard_project_init.md b/doc/nonstandard_project_init.md new file mode 100644 --- /dev/null +++ b/doc/nonstandard_project_init.md @@ -0,0 +1,103 @@ +# Non-standard project initialization + +## Introduction +The purpose of this page is to collect information about issues that arise when users either have an existing cabal project or another nonstandard setup such as a private hackage database. + +## Using a Cabal File +New users may be confused by the fact that you must add dependencies to the package's cabal file, even in the case when you have already listed the package in the `stack.yaml`. In most cases, dependencies for your package that are in the Stackage snapshot need *only* be added to the cabal file. stack makes heavy use of Cabal the library under the hood. In general, your stack packages should also end up being valid cabal-install packages. + +### Issues Referenced + - https://github.com/commercialhaskell/stack/issues/105 + +## Passing Flags to Cabal + +Any build command, `bench`, `install`, `haddock`, `test`, etc. takes a `--flag` option which passes flags to cabal. Another way to do this is using the flags field in a `stack.yaml`, with the option to specify flags on a per package basis. + +As an example, in a `stack.yaml` for multi-package project with packages `foo`, `bar`, `baz`: + +``` +flags: + foo: + release: true + bar: + default: true + baz: + manual: true +``` + +It is also possible to pass the same flag to multiple packages, i.e. `stack build --flag *:necessary` + +Currently one needs to list all of your modules that interpret flags in the `other-modules` section of a cabal file. `cabal-install` has a different behavior currently and doesn't require that the modules be listed. This may change in a future release. + + +### Issues Referenced + - https://github.com/commercialhaskell/stack/issues/191 + - https://github.com/commercialhaskell/stack/issues/417 + - https://github.com/commercialhaskell/stack/issues/335 + - https://github.com/commercialhaskell/stack/issues/301 + - https://github.com/commercialhaskell/stack/issues/365 + - https://github.com/commercialhaskell/stack/issues/105 + +## Selecting a Resolver + +`stack init` or `stack new` will try to default to the current Haskell LTS present on `https://www.stackage.org/snapshots` if no snapshot has been previously used locally, and to the latest LTS snapshot locally used for a build otherwise. Using an incorrect resolver can cause a build to fail if the version of GHC it requires is not present. + +In order to override the resolver entry at project initialization one can pass `--prefer-lts` or `--prefer-nightly`. These options will choose the latest LTS or nightly versions locally used. +Alternatively the `--resolver` option can be used with the name of any snapshots on Stackage, or with `lts` or `nightly` to select the latest versions, disregarding previously used ones. This is not the default so as to avoid unnecessary recompilation time. + +:TODO: Document `--solver` + +### Issues Referenced + - https://github.com/commercialhaskell/stack/issues/468 + - https://github.com/commercialhaskell/stack/issues/464 + +## Using git Repositories +stack has support for packages that reside in remote git locations. + +Example: + +``` +packages: +- '.' +- location: + git: https://github.com/kolmodin/binary + commit: 8debedd3fcb6525ac0d7de2dd49217dce2abc0d9 +``` + +### Issues Referenced + - https://github.com/commercialhaskell/stack/issues/254 + - https://github.com/commercialhaskell/stack/issues/199 + +## Private Hackage +Working with a private Hackage is currently supported in certain situations. +There exist special entries in `stack.yaml` that may help you. In a `stack.yaml` file, it is possible +to add lines for packages in your database referencing the sdist locations via an `http` entry, or to use a `Hackage` entry. + +The recommended stack workflow is to use git submodules instead of a private Hackage. Either by using git submodules and listing the directories in the packages section of `stack.yaml`, or by adding the private dependencies as git URIs with a commit SHA to the `stack.yaml`. This has the large benefit of eliminating the need to manage a Hackage database and pointless version bumps. + +For further information see [YAML configuration](yaml_configuration.md) + +### Issues Referenced + - https://github.com/commercialhaskell/stack/issues/445 + - https://github.com/commercialhaskell/stack/issues/565 + +## Custom Snapshots +Currently WIP? +### Issues Referenced + - https://github.com/commercialhaskell/stack/issues/111 + - https://github.com/commercialhaskell/stack/issues/253 + - https://github.com/commercialhaskell/stack/issues/137 + +## Intra-package Targets +stack supports intra-package targets, similar to `cabal build COMPONENTS` for situations when you don't want to build every target inside your package. + +Example: +``` +stack build stack:lib:stack +stack test stack:test:stack-integration-test +``` + +Note: this does require prefixing the component name with the package name. + +### Issues referenced + - https://github.com/commercialhaskell/stack/issues/201 diff --git a/doc/shell_autocompletion.md b/doc/shell_autocompletion.md new file mode 100644 --- /dev/null +++ b/doc/shell_autocompletion.md @@ -0,0 +1,35 @@ +# Shell Auto-completion + +Note: if you installed a package for you Linux distribution, the bash completion +file was automatically installed (you may need the `bash-completion` package to +have it take effect). + +The following adds support for shell tab completion for standard Stack arguments, although completion for filenames and executables etc. within stack is still lacking (see [issue 823](https://github.com/commercialhaskell/stack/issues/832)). + +## for bash users + +you need to run following command +``` +eval "$(stack --bash-completion-script stack)" +``` +You can also add it to your `.bashrc` file if you want. + +## for ZSH users + +documentation says: +> Zsh can handle bash completions functions. The latest development version of zsh has a function bashcompinit, that when run will allow zsh to read bash completion specifications and functions. This is documented in the zshcompsys man page. To use it all **you need to do is run bashcompinit at any time after compinit**. It will define complete and compgen functions corresponding to the bash builtins. + +You must so: + 1. launch compinint + 2. launch bashcompinit + 3. eval stack bash completion script + +```shell +autoload -U +X compinit && compinit +autoload -U +X bashcompinit && bashcompinit +eval "$(stack --bash-completion-script stack)" +``` + +:information_source: If you already have quite a large zshrc, or if you use oh-my-zsh, **compinit** will probably already be loaded. If you have a blank zsh config, all of the 3 lines above are necessary. + +:gem: tip: instead of running those 3 lines from your shell every time you want to use stack, you can add those 3 lines in your $HOME/.zshrc file diff --git a/doc/travis_ci.md b/doc/travis_ci.md new file mode 100644 --- /dev/null +++ b/doc/travis_ci.md @@ -0,0 +1,151 @@ +# Travis CI + +For many use cases, the +[Travis caching section of the user guide](GUIDE.md#travis-with-caching) +will be sufficient. + +This page documents how to use Stack on [Travis CI](https://travis-ci.org/). We +assume you have basic familiarity with Travis. + +*Note:* both Travis and Stack infrastructures are actively developed. We try to + document best practices at the moment. + +## Container infrastructure + +For Stack on Travis to be practical, we must use caching. Otherwise build times +will take an incredibly long time, about 30 minutes versus 3-5. Caching is +currently available only for +[container-based Travis infrastructure](http://docs.travis-ci.com/user/workers/container-based-infrastructure/). +Shortly we have to add + +```yaml +sudo: false + +# Caching so the next build will be fast too. +cache: + directories: + - $HOME/.stack +``` + +To the `.travis.yml`. This however restricts how we can install GHC and Stack on +the Travis machines. + +## Installing Stack + +Currently there is only one reasonable way to install Stack: fetch precompiled +binary from the Github. + +```yaml +before_install: +# Download and unpack the stack executable +- mkdir -p ~/.local/bin +- export PATH=$HOME/.local/bin:$PATH +- travis_retry curl -L https://www.stackage.org/stack/linux-x86_64 | tar xz --wildcards --strip-components=1 -C ~/.local/bin '*/stack' +``` + +Once Travis whitelists the stack .deb files, we'll be able to simply include +stack in the `addons` section, and automatically use the newest version of +stack, avoiding that complicated `before_install` section This is being +tracked in the +[apt-source-whitelist](https://github.com/travis-ci/apt-source-whitelist/pull/7) +and +[apt-package-whitelist](https://github.com/travis-ci/apt-package-whitelist/issues/379) +issue trackers. + +## Installing GHC + +There are two ways to install GHC: + +- Let Stack download GHC +- Install GHC using [apt plugin](http://docs.travis-ci.com/user/apt/) + +See the +[Travis caching section of the user guide](GUIDE.md#travis-with-caching) for +an example of the first option (letting Stack download GHC). Here, we will +explain the second option. With single GHC the situation is simple: + +```yaml +before_install: + - export PATH=/opt/ghc/7.10.2/bin:$PATH + +addons: + apt: + sources: + - hvr-ghc + packages: + - ghc-7.10.2 +``` + +### Multiple GHC - parametrised builds + +Travis apt plugin doesn't yet support installing apt packages dynamically +(https://github.com/travis-ci/travis-ci/issues/4291). That for we need to write +a bit repetitive `.travis.yml`. + +Also for different GHC versions, you probably want to use different `stack.yaml` +files. + +```yaml +# N.B. No top-level env: declaration! + +matrix: + include: + - env: GHCVER=7.8.4 STACK_YAML=stack.yaml + addons: + apt: + sources: + - hvr-ghc + packages: + - ghc-7.8.4 + - env: GHCVER=7.10.1 STACK_YAML=stack-7.10.yaml + addons: + apt: + sources: + - hvr-ghc + packages: + - ghc-7.10.1 + - env: GHCVER=head STACK_YAML=stack-head.yaml + addons: + apt: + sources: + - hvr-ghc + packages: + - ghc-head + allow_failures: + - env: GHCVER=head STACK_YAML=stack-head.yaml + +before_install: + # ghc + - export PATH=/opt/ghc/$GHCVER/bin:$PATH +``` + +Especially to use ghc `HEAD` you need to pass `--skip-ghc-check` option to Stack. + +## Running tests + +After the environment setup, actual test running is simple: + +```yaml +script: + - stack --no-terminal --skip-ghc-check test +``` + +## Other details + +Some Stack commands will run for long time (when cache is cold) without +producing any output. For Travis not to timeout, one can wrap commands in +[a simple script](https://github.com/futurice/fum2github/blob/master/travis_long). + +```yaml +install: + - ./travis_long stack --no-terminal --skip-ghc-check setup + - ./travis_long stack --no-terminal --skip-ghc-check test --only-snapshot +``` + +## Examples + +- [futurice/fum2github](https://github.com/futurice/fum2github/blob/master/.travis.yml) +- [haskell-distributed/cloud-haskell](https://github.com/haskell-distributed/cloud-haskell/blob/master/.travis.yml) +- [simonmichael/hledger](https://github.com/simonmichael/hledger/blob/master/.travis.yml) +- [fpco/wai-middleware-crowd](https://github.com/fpco/wai-middleware-crowd/blob/master/.travis.yml) +- [commercialhaskell/all-cabal-hashes-tool](https://github.com/commercialhaskell/all-cabal-hashes-tool/blob/master/.travis.yml) diff --git a/doc/yaml_configuration.md b/doc/yaml_configuration.md new file mode 100644 --- /dev/null +++ b/doc/yaml_configuration.md @@ -0,0 +1,539 @@ +# YAML Configuration + +This page is intended to fully document all configuration options available in the stack.yaml file. Note that this page is likely to be both *incomplete* and sometimes *inaccurate*. If you see such cases, please update the page, and if you're not sure how, open an issue labeled "question". + +The stack.yaml configuration options break down into [project specific](#project-config) options in: + +- `/stack.yaml` + +and [non-project specific](#non-project-config) options in: + +- `/etc/stack/config.yaml` -- for system global non-project default options +- `~/.stack/config.yaml` -- for user non-project default options +- The project file itself may also contain non-project specific options + +*Note:* When stack is invoked outside a stack project it will source project specific options from `~/.stack/global/stack.yaml`. Options in this file will be ignored for a project with its own `/stack.yaml`. + +## Project config + +Project specific options are only valid in the `stack.yaml` file local to a project, not in the user or global config files. + +### packages + +The `packages` section lists all local (project) packages. The term _local +package_ should be differentiated from a _dependency package_. A local package +is something that you are developing as part of the project. Whereas a +dependency package is an external package that your project depends on. + +In its simplest usage, it will be a list of directories or HTTP(S) URLs to a +tarball or a zip. For example: + +```yaml +packages: + - . + - dir1/dir2 + - https://example.com/foo/bar/baz-0.0.2.tar.gz +``` + +Each package directory or location specified must have a valid cabal file +present. Note that the subdirectories of the directory are not searched for +cabal files. Subdirectories will have to be specified as independent items in +the list of packages. + +When the `packages` field is not present, it defaults to looking for a package +in the project's root directory: + +```yaml +packages: + - . +``` +#### Complex package locations (`location`) + +More complex package locations can be specified in a key-value format with +`location` as a mandatory key. In addition to `location` some optional +key-value pairs can be specified to include specific subdirectories or to +specify package attributes as descibed later in this section. + +In its simplest form a `location` key can have a single value in the same way +as described above for single value items. Alternativel it can have key-value +pairs as subfields to describe a git or mercurial repository location. For +example: + +```yaml +packages: +- location: . +- location: dir1/dir2 +- location: https://example.com/foo/bar/baz-0.0.2.tar.gz +- location: http://github.com/yesodweb/wai/archive/2f8a8e1b771829f4a8a77c0111352ce45a14c30f.zip +- location: + git: git@github.com:commercialhaskell/stack.git + commit: 6a86ee32e5b869a877151f74064572225e1a0398 +- location: + hg: https://example.com/hg/repo + commit: da39a3ee5e6b4b0d3255bfef95601890afd80709 +``` + +Note: it is highly recommended that you only use SHA1 values for a Git or +Mercurial commit. Other values may work, but they are not officially supported, +and may result in unexpected behavior (namely, stack will not automatically +pull to update to new versions). + +A `location` key can be accompanied by a `subdirs` key to look for cabal files +in a list of subdirectories as well in addition to the top level directory. + +This could be useful for mega-repos like +[wai](https://github.com/yesodweb/wai/) or +[digestive-functors](https://github.com/jaspervdj/digestive-functors). + +The `subdirs` key can have multiple nested series items specifying a list of +subdirectories. For example: +```yaml +packages: +- location: . + subdirs: + - subdir1 + - subdir2 +- location: + git: git@github.com:yesodweb/wai + commit: 2f8a8e1b771829f4a8a77c0111352ce45a14c30f + subdirs: + - auto-update + - wai +- location: http://github.com/yesodweb/wai/archive/2f8a8e1b771829f4a8a77c0111352ce45a14c30f.zip + subdirs: + - auto-update + - wai +``` + +#### Local dependency packages (`extra-dep`) +A `location` key can be accompanied by an `extra-dep` key. When the +`extra-dep` key is set to `true` it indicates that the package should be +treated in the same way as a dependency package and not as part of the project. +This means the following: +* A _dependency package_ is built only if a user package or its dependencies + depend on it. Note that a regular _project package_ is built anyway even if + no other package depends on it. +* Its test suites and benchmarks will not be run. +* It will not be directly loaded in ghci when `stack ghci` is run. This is + important because if you specify huge dependencies as project packages then + ghci will have a nightmare loading everything. + +This is especially useful when you are tweaking upstream packages or want to +use latest versions of the upstream packages which are not yet on Hackage or +Stackage. + +For example: +```yaml +packages: +- location: . +- location: vendor/binary + extra-dep: true +- location: + git: git@github.com:yesodweb/wai + commit: 2f8a8e1b771829f4a8a77c0111352ce45a14c30f + subdirs: + - auto-update + - wai + extra-dep: true +``` + +### extra-deps + +This is a list of package identifiers for additional packages from upstream to +be included. This is usually used to augment an LTS Haskell or Stackage Nightly +snapshot with a package that is not present or is at an different version than you +wish to use. + +```yaml +extra-deps: +- acme-missiles-0.3 +``` + +Note that the `extra-dep` attribute in the `packages` section as described in +an earlier section is used for non-index local or remote packages while the +`extra-deps` section is for packages to be automatically pulled from an index +like Hackage. + +### resolver + +Specifies how dependencies are resolved. There are currently four resolver types: + +* LTS Haskell snapshots, e.g. `resolver: lts-2.14` +* Stackage Nightly snapshot, e.g. `resolver: nightly-2015-06-16` +* No snapshot, just use packages shipped with the compiler + * For GHC this looks like `resolver: ghc-7.10.2` + * For GHCJS this looks like `resolver: ghcjs-0.1.0_ghc-7.10.2`. +* [Custom snapshot](https://github.com/commercialhaskell/stack/wiki/Custom-Snapshot) + +Each of these resolvers will also determine what constraints are placed on the compiler version. See the [compiler-check](#compiler-check) option for some additional control over compiler version. + +### flags + +Flags can be set for each package separately, e.g. + +```yaml +flags: + package-name: + flag-name: true +``` + +Flags will only affect packages in your `packages` and `extra-deps` settings. +Packages that come from the snapshot global database are not affected. + +### image + +The image settings are used for the creation of container images using `stack +image container`, e.g. + +```yaml +image: + containers: + - base: "fpco/stack-build" + add: + static: /data/static +``` + +`base` is the docker image that will be used to built upon. The `add` lines +allow you to add additional directories to your image. You can specify the name +of the image using `name` (otherwise it defaults to the same as your project). +You can also specify `entrypoints`. By default all your executables are placed +in `/usr/local/bin`, but you can specify a list using `executables` to only add +some. + +### user-message + +A user-message is inserted by `stack init` when it omits packages or adds +external dependencies. For example: + +```yaml +user-message: ! 'Warning: Some packages were found to be incompatible with the resolver + and have been left commented out in the packages section. + + Warning: Specified resolver could not satisfy all dependencies. Some external packages + have been added as dependencies. + + You can suppress this message by removing it from stack.yaml + +' +``` + +This messages is displayed every time the config is loaded by stack and serves +as a reminder for the user to review the configuration and make any changes if +needed. The user can delete this message if the generated configuration is +acceptable. + +## Non-project config + +Non-project config options may go in the global config (`/etc/stack/config.yaml`) or the user config (`~/.stack/config.yaml`). + +### docker + +See [Docker integration](docker_integration.md#configuration). + +### nix + +(since 0.1.10.0) + +See [Nix integration](nix_integration.md#configuration). + +### connection-count + +Integer indicating how many simultaneous downloads are allowed to happen + +Default: `8` + +### hide-th-loading + +Strip out the "Loading ..." lines from GHC build output, produced when using Template Haskell + +Default: `true` + +### latest-snapshot-url + +URL providing a JSON with information on the latest LTS and Nightly snapshots, used for automatic project configuration. + +Default: `https://www.stackage.org/download/snapshots.json` + +### local-bin-path + +Target directory for `stack install` and `stack build --copy-bins`. + +Default: `~/.local/bin` + +### package-indices + +```yaml +package-indices: +- name: Hackage + download-prefix: https://s3.amazonaws.com/hackage.fpcomplete.com/package/ + + # at least one of the following must be present + git: https://github.com/commercialhaskell/all-cabal-hashes.git + http: https://s3.amazonaws.com/hackage.fpcomplete.com/00-index.tar.gz + + # optional fields, both default to false + gpg-verify: false + require-hashes: false +``` + +One thing you should be aware of: if you change the contents of package-version +combination by setting a different package index, this *can* have an effect on +other projects by installing into your shared snapshot database. + +### system-ghc + +Enables or disables using the GHC available on the PATH. Useful to disable if +you want to force stack to use its own installed GHC (via `stack setup`), in +cases where your system GHC my be incomplete for some reason. Default is `true`. + +```yaml +# Turn off system GHC +system-ghc: false +``` + +### install-ghc + +Whether or not to automatically install GHC when necessary. Default is `false`, +which means stack will prompt you to run `stack setup` as needed. + +### skip-ghc-check + +Should we skip the check to confirm that your system GHC version (on the PATH) matches what your project expects? Default is `false`. + +### require-stack-version + +Require a version of stack within the specified range +([cabal-style](https://www.haskell.org/cabal/users-guide/developing-packages.html#build-information)) +to be used for this project. Example: `require-stack-version: "== 0.1.*"` + +Default: `"-any"` + +### arch/os + +Set the architecture and operating system for GHC, build directories, etc. Values are those recognized by Cabal, e.g.: + + arch: i386, x86_64 + os: windows, linux + +You likely only ever want to change the arch value. This can also be set via the command line. + +### extra-include-dirs/extra-lib-dirs + +A list of extra paths to be searched for header files and libraries, respectively. Paths should be absolute + +```yaml +extra-include-dirs: +- /opt/foo/include +extra-lib-dirs: +- /opt/foo/lib +``` + +### compiler-check + +(Since 0.1.4) + +Specifies how the compiler version in the resolver is matched against concrete versions. Valid values: + +* `match-minor`: make sure that the first three components match, but allow patch-level differences. For example< 7.8.4.1 and 7.8.4.2 would both match 7.8.4. This is useful to allow for custom patch levels of a compiler. This is the default +* `match-exact`: the entire version number must match precisely +* `newer-minor`: the third component can be increased, e.g. if your resolver is `ghc-7.10.1`, then 7.10.2 will also be allowed. This was the default up through stack 0.1.3 + +### compiler + +(Since 0.1.7) + +Overrides the compiler version in the resolver. Note that the `compiler-check` +flag also applies to the version numbers. This uses the same syntax as compiler +resolvers like `ghc-7.10.2` or `ghcjs-0.1.0.20150924_ghc-7.10.2` (version used +for the 'old-base' version of GHCJS). While it's useful to override the +compiler for a variety of reasons, the main usecase is to use GHCJS with a +stackage snapshot, like this: + +```yaml +resolver: lts-3.10 +compiler: ghcjs-0.1.0.20150924_ghc-7.10.2 +compiler-check: match-exact +``` + +### ghc-options + +(Since 0.1.4) + +Allows specifying per-package and global GHC options: + +```yaml +ghc-options: + # All packages + "*": -Wall + some-package: -DSOME_CPP_FLAG +``` + +Caveat emptor: setting options like this will affect your snapshot packages, +which can lead to unpredictable behavior versus official Stackage snapshots. +This is in contrast to the `ghc-options` command line flag, which will only +affect local packages. + +### ghc-variant + +(Since 0.1.5) + +Specify a variant binary distribution of GHC to use. Known values: + +* `standard`: This is the default, uses the standard GHC binary distribution +* `gmp4`: Use the "centos6" GHC bindist, for Linux systems with libgmp4 (aka + `libgmp.so.3`), such as CentOS 6. This variant will be used automatically on such systems; you should not need to specify it in the configuration +* `integersimple`: Use a GHC bindist that uses + [integer-simple instead of GMP](https://ghc.haskell.org/trac/ghc/wiki/ReplacingGMPNotes) +* any other value: Use a custom GHC bindist. You should specify + [setup-info](#setup-info) so `stack setup` knows where to download it, or + pass the `stack setup --ghc-bindist` argument on the command-line + +### setup-info + +(Since 0.1.5) + +Allows overriding from where tools like GHC and msys2 (on Windows) are +downloaded. Most useful for specifying locations of custom GHC binary +distributions (for use with the [ghc-variant](#ghc-variant) option): + +```yaml +setup-info: + ghc: + windows32-custom-foo: + 7.10.2: + url: "https://example.com/ghc-7.10.2-i386-unknown-mingw32-foo.tar.xz" +``` + +### pvp-bounds + +(Since 0.1.5) + +When using the `sdist` and `upload` commands, this setting determines whether +the cabal file's dependencies should be modified to reflect PVP lower and upper +bounds. Values are `none` (unchanged), `upper` (add upper bounds), `lower` (add +lower bounds), and both (and upper and lower bounds). The algorithm it follows +is: + +* If an upper or lower bound already exists on a dependency, it's left alone +* When adding a lower bound, we look at the current version specified by stack.yaml, and set it as the lower bound (e.g., `foo >= 1.2.3`) +* When adding an upper bound, we require less than the next major version (e.g., `foo < 1.3`) + +```yaml +pvp-bounds: none +``` + +For more information, see [the announcement blog post](https://www.fpcomplete.com/blog/2015/09/stack-pvp). + +### modify-code-page + +(Since 0.1.6) + +Modify the code page for UTF-8 output when running on Windows. Default behavior +is to modify. + +```yaml +modify-code-page: false +``` + +### explicit-setup-deps + +(Since 0.1.6) + +Decide whether a custom `Setup.hs` script should be run with an explicit list of +dependencies, based on the dependencies of the package itself. It associates the +name of a local package with a boolean. When it's `true`, the `Setup.hs` script +is built with an explicit list of packages. When it's `false` (default), the +`Setup.hs` script is built without access to the local DB, but can access any +package in the snapshot / global DB. + +Note that in the future, this will be unnecessary, once Cabal provides full +support for explicit Setup.hs dependencies. + +```yaml +explicit-setup-deps: + "*": true # change the default + entropy: false # override the new default for one package +``` + +### rebuild-ghc-options + +(Since 0.1.6) + +Should we rebuild a package when its GHC options change? Before 0.1.6, this was a non-configurable true. However, in most cases, the flag is used to affect optimization levels and warning behavior, for which GHC itself doesn't actually recompile the modules anyway. Therefore, the new behavior is to not recompile on an options change, but this behavior can be changed back with the following: + +```yaml +rebuild-ghc-options: true +``` + +### apply-ghc-options + +(Since 0.1.6) + +Which packages do ghc-options on the command line get applied to? Before 0.1.6, the default value was `targets` + +```yaml +apply-ghc-options: locals # all local packages, the default +# apply-ghc-options: targets # all local packages that are targets +# apply-ghc-options: everything # applied even to snapshot and extra-deps +``` + +Note that `everything` is a slightly dangerous value, as it can break invariants about your snapshot database. + +### allow-newer + +(Since 0.1.7) + +Ignore version bounds in .cabal files. Default is false. + +```yaml +allow-newer: true +``` + +Note that this also ignores lower bounds. The name "allow-newer" is chosen to +match the commonly used cabal option. + +### allow-different-user + +(Since 1.0.1) + +Allow users other than the owner of the stack root directory (typically `~/.stack`) +to use the stack installation. The default is `false`. POSIX systems only. + +```yaml +allow-different-user: true +``` + +The intention of this option is to prevent file permission problems, for example +as the result of a `stack` command executed under `sudo`. + +The option is automatically enabled when `stack` is re-spawned in a Docker process. + +### templates + +Templates used with `stack new` have a number of parameters that affect the generated code. These can be set for all new projects you create. The result of them can be observed in the generated LICENSE and cabal files. + +The 5 parameters are: `author-email`, `author-name`, `category`, `copyright` and `github-username`. + +* _author-email_ - sets the `maintainer` property in cabal +* _author-name_ - sets the `author` property in cabal and the name used in LICENSE +* _category_ - sets the `category` property in cabal. This is used in Hackage. For examples of categories see [Packages by category](https://hackage.haskell.org/packages/). It makes sense for `category` to be set on a per project basis because it is uncommon for all projects a user creates to belong to the same category. The category can be set per project by passing `-p "category:value"` to the `stack new` command. +* _copyright_ - sets the `copyright` property in cabal. It is typically the name of the holder of the copyright on the package and the year(s) from which copyright is claimed. For example: `Copyright: (c) 2006-2007 Joe Bloggs` +* _github-username_ - used to generate `homepage` and `source-repository` in cabal. For instance `github-username: myusername` and `stack new my-project new-template` would result: +```yaml +homepage: http://github.com/myusername/my-project#readme + +source-repository head + type: git + location: https://github.com/myusername/my-project +``` + +These properties can be set in `config.yaml` as follows: +```yaml +templates: + params: + author-name: Your Name + author-email: youremail@example.com + category: Your Projects Category + copyright: 'Copyright: (c) 2016 Your Name' + github-username: yourusername +``` diff --git a/src/Data/Attoparsec/Interpreter.hs b/src/Data/Attoparsec/Interpreter.hs --- a/src/Data/Attoparsec/Interpreter.hs +++ b/src/Data/Attoparsec/Interpreter.hs @@ -60,22 +60,23 @@ import Data.Conduit import Data.Conduit.Attoparsec import qualified Data.Conduit.Binary as CB -import Data.Conduit.Text(decodeUtf8) +import Data.Conduit.Text (decodeUtf8) import Data.List (intercalate) import Data.Text (pack) import Stack.Constants +import System.FilePath (takeExtension) import System.IO (IOMode (ReadMode), withBinaryFile, stderr, hPutStrLn) -- | Parser to extract the stack command line embedded inside a comment -- after validating the placement and formatting rules for a valid -- interpreter specification. -interpreterArgsParser :: String -> P.Parser String -interpreterArgsParser progName = P.option "" sheBangLine *> interpreterComment +interpreterArgsParser :: Bool -> String -> P.Parser String +interpreterArgsParser isLiterate progName = P.option "" sheBangLine *> interpreterComment where sheBangLine = P.string "#!" *> P.manyTill P.anyChar P.endOfLine - commentStart str = (P.string str (progName ++ " options comment")) + commentStart psr = (psr (progName ++ " options comment")) *> P.skipSpace *> (P.string (pack progName) show progName) @@ -87,10 +88,25 @@ *> ((end >> return "") <|> (P.space *> (P.manyTill anyCharNormalizeSpace end "-}"))) + horizontalSpace = P.satisfy P.isHorizontalSpace + lineComment = comment "--" (P.endOfLine <|> P.endOfInput) + literateLineComment = comment + (">" *> horizontalSpace *> "--") + (P.endOfLine <|> P.endOfInput) blockComment = comment "{-" (P.string "-}") - interpreterComment = lineComment <|> blockComment + literateBlockComment = + (">" *> horizontalSpace *> "{-") + *> P.skipMany (("" <$ horizontalSpace) <|> (P.endOfLine *> ">")) + *> (P.string (pack progName) progName) + *> (P.manyTill' (P.satisfy (not . P.isEndOfLine) + <|> (' ' <$ (P.endOfLine *> ">" ">"))) "-}") + + interpreterComment = if isLiterate + then literateLineComment <|> literateBlockComment + else lineComment <|> blockComment + -- | Extract stack arguments from a correctly placed and correctly formatted -- comment when it is being used as an interpreter getInterpreterArgs :: String -> IO [String] @@ -103,7 +119,9 @@ parseFile h = CB.sourceHandle h =$= decodeUtf8 - $$ sinkParserEither (interpreterArgsParser stackProgName) + $$ sinkParserEither (interpreterArgsParser isLiterate stackProgName) + + isLiterate = takeExtension file == ".lhs" -- FIXME We should print anything only when explicit verbose mode is -- specified by the user on command line. But currently the diff --git a/src/Data/Set/Monad.hs b/src/Data/Set/Monad.hs deleted file mode 100644 --- a/src/Data/Set/Monad.hs +++ /dev/null @@ -1,28 +0,0 @@ --- | Monadic operations for 'Set'. - -module Data.Set.Monad - (mapM - ,mapM_ - ,filterM) - where - -import Control.Monad (liftM) -import qualified Control.Monad as L -import Data.Set (Set) -import qualified Data.Set as S -import Prelude hiding (mapM,mapM_) - --- | Map over a 'Set' in a monad. -mapM :: (Ord a,Ord b,Monad m) - => (a -> m b) -> Set a -> m (Set b) -mapM f = liftM S.fromList . L.mapM f . S.toList - --- | Map over a 'Set' in a monad, discarding the result. -mapM_ :: (Ord a,Ord b,Monad m) - => (a -> m b) -> Set a -> m () -mapM_ f = L.mapM_ f . S.toList - --- | Filter elements of a 'Set' in a monad. -filterM :: (Ord a,Monad m) - => (a -> m Bool) -> Set a -> m (Set a) -filterM f = liftM S.fromList . L.filterM f . S.toList diff --git a/src/Network/HTTP/Download.hs b/src/Network/HTTP/Download.hs --- a/src/Network/HTTP/Download.hs +++ b/src/Network/HTTP/Download.hs @@ -1,3 +1,4 @@ +{-# LANGUAGE CPP #-} {-# LANGUAGE DeriveDataTypeable #-} {-# LANGUAGE MultiParamTypeClasses #-} {-# LANGUAGE OverloadedStrings #-} @@ -25,7 +26,7 @@ import Control.Exception (Exception) import Control.Exception.Enclosed (handleIO) import Control.Monad (void) -import Control.Monad.Catch (MonadThrow, throwM) +import Control.Monad.Catch (MonadThrow, MonadMask, throwM) import Control.Monad.IO.Class (MonadIO, liftIO) import Control.Monad.Reader (MonadReader, ReaderT, ask, runReaderT) @@ -102,37 +103,39 @@ } req2 = req1 { checkStatus = \_ _ _ -> Nothing } env <- ask - liftIO $ flip runReaderT env $ withResponse req2 $ \res -> case () of - () - | responseStatus res == status200 -> liftIO $ do - createDirectoryIfMissing True $ takeDirectory destFilePath + liftIO $ recoveringHttp drRetryPolicyDefault $ flip runReaderT env $ + withResponse req2 $ \res -> case () of + () + | responseStatus res == status200 -> liftIO $ do + createDirectoryIfMissing True $ takeDirectory destFilePath - -- Order here is important: first delete the etag, then write the - -- file, then write the etag. That way, if any step fails, it will - -- force the download to happen again. - handleIO (const $ return ()) $ removeFile etagFilePath + -- Order here is important: first delete the etag, then write the + -- file, then write the etag. That way, if any step fails, it will + -- force the download to happen again. + handleIO (const $ return ()) $ removeFile etagFilePath - let destFilePathTmp = destFilePath <.> "tmp" - withBinaryFile destFilePathTmp WriteMode $ \h -> - responseBody res $$ sinkHandle h - renameFile destFilePathTmp destFilePath + let destFilePathTmp = destFilePath <.> "tmp" + withBinaryFile destFilePathTmp WriteMode $ \h -> + responseBody res $$ sinkHandle h + renameFile destFilePathTmp destFilePath - forM_ (lookup "ETag" (responseHeaders res)) $ \e -> do - let tmp = etagFilePath <.> "tmp" - S.writeFile tmp e - renameFile tmp etagFilePath + forM_ (lookup "ETag" (responseHeaders res)) $ \e -> do + let tmp = etagFilePath <.> "tmp" + S.writeFile tmp e + renameFile tmp etagFilePath - return True - | responseStatus res == status304 -> return False - | otherwise -> throwM $ RedownloadFailed req2 dest $ void res + return True + | responseStatus res == status304 -> return False + | otherwise -> throwM $ RedownloadFailed req2 dest $ void res -- | Download a JSON value and parse it using a 'FromJSON' instance. -downloadJSON :: (FromJSON a, MonadReader env m, HasHttpManager env, MonadIO m, MonadThrow m) +downloadJSON :: (FromJSON a, MonadReader env m, HasHttpManager env, MonadIO m, MonadThrow m, MonadMask m) => Request -> m a downloadJSON req = do - val <- liftHTTP $ withResponse req $ \res -> - responseBody res $$ sinkParser json' + val <- recoveringHttp drRetryPolicyDefault $ + liftHTTP $ withResponse req $ \res -> + responseBody res $$ sinkParser json' case parseEither parseJSON val of Left e -> throwM $ DownloadJSONException req e Right x -> return x diff --git a/src/Network/HTTP/Download/Verified.hs b/src/Network/HTTP/Download/Verified.hs --- a/src/Network/HTTP/Download/Verified.hs +++ b/src/Network/HTTP/Download/Verified.hs @@ -10,6 +10,7 @@ {-# LANGUAGE StandaloneDeriving #-} module Network.HTTP.Download.Verified ( verifiedDownload + , recoveringHttp , DownloadRequest(..) , drRetryPolicyDefault , HashCheck(..) @@ -179,6 +180,28 @@ hashChecksToZipSink :: MonadThrow m => Request -> [HashCheck] -> ZipSink ByteString m () hashChecksToZipSink req = traverse_ (ZipSink . sinkCheckHash req) +-- 'Control.Retry.recovering' customized for HTTP failures +recoveringHttp :: (MonadMask m, MonadIO m) + => RetryPolicy -> m a -> m a +recoveringHttp retryPolicy = +#if MIN_VERSION_retry(0,7,0) + recovering retryPolicy handlers . const +#else + recovering retryPolicy handlers +#endif + where + handlers = [const $ Handler alwaysRetryHttp,const $ Handler retrySomeIO] + + alwaysRetryHttp :: Monad m => HttpException -> m Bool + alwaysRetryHttp _ = return True + + retrySomeIO :: Monad m => IOException -> m Bool + retrySomeIO e = return $ case ioe_type e of + -- hGetBuf: resource vanished (Connection reset by peer) + ResourceVanished -> True + -- conservatively exclude all others + _ -> False + -- | Copied and extended version of Network.HTTP.Download.download. -- -- Has the following additional features: @@ -203,27 +226,11 @@ liftIO $ whenM' getShouldDownload $ do createDirectoryIfMissing True dir withBinaryFile fptmp WriteMode $ \h -> -#if MIN_VERSION_retry(0,7,0) - recovering drRetryPolicy handlers $ const $ -#else - recovering drRetryPolicy handlers $ -#endif + recoveringHttp drRetryPolicy $ flip runReaderT env $ withResponse req (go h) renameFile fptmp fp where - handlers = [const $ Handler alwaysRetryHttp,const $ Handler retrySomeIO] - - alwaysRetryHttp :: Monad m => HttpException -> m Bool - alwaysRetryHttp _ = return True - - retrySomeIO :: Monad m => IOException -> m Bool - retrySomeIO e = return $ case ioe_type e of - -- hGetBuf: resource vanished (Connection reset by peer) - ResourceVanished -> True - -- conservatively exclude all others - _ -> False - whenM' mp m = do p <- mp if p then m >> return True else return False diff --git a/src/Options/Applicative/Complicated.hs b/src/Options/Applicative/Complicated.hs --- a/src/Options/Applicative/Complicated.hs +++ b/src/Options/Applicative/Complicated.hs @@ -30,6 +30,8 @@ -> Maybe String -- ^ version string -> String + -- ^ hpack numeric version, as string + -> String -- ^ header -> String -- ^ program description @@ -41,7 +43,7 @@ -> EitherT b (Writer (Mod CommandFields (b,a))) () -- ^ commands (use 'addCommand') -> IO (a,b) -complicatedOptions numericVersion versionString h pd commonParser mOnFailure commandParser = +complicatedOptions numericVersion versionString numericHpackVersion h pd commonParser mOnFailure commandParser = do args <- getArgs (a,(b,c)) <- case execParserPure (prefs noBacktrack) parser args of Failure _ | null args -> withArgs ["--help"] (execParser parser) @@ -54,7 +56,7 @@ versionOptions = case versionString of Nothing -> versionOption (showVersion numericVersion) - Just s -> versionOption s <*> numericVersionOption + Just s -> versionOption s <*> numericVersionOption <*> numericHpackVersionOption versionOption s = infoOption s @@ -65,6 +67,11 @@ (showVersion numericVersion) (long "numeric-version" <> help "Show only version number") + numericHpackVersionOption = + infoOption + numericHpackVersion + (long "hpack-numeric-version" <> + help "Show only hpack's version number") -- | Add a command to the options dispatcher. addCommand :: String -- ^ command string diff --git a/src/Path/Extra.hs b/src/Path/Extra.hs --- a/src/Path/Extra.hs +++ b/src/Path/Extra.hs @@ -7,10 +7,16 @@ ,dropRoot ,parseCollapsedAbsDir ,parseCollapsedAbsFile + ,rejectMissingFile + ,rejectMissingDir ) where +import Control.Monad (liftM) import Control.Monad.Catch +import Control.Monad.IO.Class +import Data.Bool (bool) import Path +import Path.IO import Path.Internal (Path(..)) import qualified System.FilePath as FP @@ -60,3 +66,31 @@ -- Windows). dropRoot :: Path Abs t -> Path Rel t dropRoot (Path l) = Path (FP.dropDrive l) + +-- | If given file in 'Maybe' does not exist, ensure we have 'Nothing'. This +-- is to be used in conjunction with 'forgivingAbsence' and +-- 'resolveFile'. +-- +-- Previously the idiom @forgivingAbsence (relsoveFile …)@ alone was used, +-- which relied on 'canonicalizePath' throwing 'isDoesNotExistError' when +-- path does not exist. As it turns out, this behavior is actually not +-- intentional and unreliable, see +-- . This was “fixed” in +-- version @1.2.3.0@ of @directory@ package (now it never throws). To make +-- it work with all versions, we need to use the following idiom: +-- +-- > forgivingAbsence (resolveFile …) >>= rejectMissingFile + +rejectMissingFile :: MonadIO m + => Maybe (Path Abs File) + -> m (Maybe (Path Abs File)) +rejectMissingFile Nothing = return Nothing +rejectMissingFile (Just p) = bool Nothing (Just p) `liftM` doesFileExist p + +-- | See 'rejectMissingFile'. + +rejectMissingDir :: MonadIO m + => Maybe (Path Abs Dir) + -> m (Maybe (Path Abs Dir)) +rejectMissingDir Nothing = return Nothing +rejectMissingDir (Just p) = bool Nothing (Just p) `liftM` doesDirExist p diff --git a/src/Path/Find.hs b/src/Path/Find.hs --- a/src/Path/Find.hs +++ b/src/Path/Find.hs @@ -9,6 +9,8 @@ ,findInParents) where +import Control.Exception (evaluate) +import Control.DeepSeq (force) import Control.Monad import Control.Monad.Catch import Control.Monad.IO.Class @@ -16,6 +18,7 @@ import Data.List import Path import Path.IO hiding (findFiles) +import System.PosixCompat.Files (getSymbolicLinkStatus, isSymbolicLink) -- | Find the location of a file matching the given predicate. findFileUp :: (MonadIO m,MonadThrow m) @@ -50,6 +53,12 @@ | otherwise -> findPathUp pathType (parent dir) p upperBound -- | Find files matching predicate below a root directory. +-- +-- NOTE: this skips symbolic directory links, to avoid loops. This may +-- not make sense for all uses of file finding. +-- +-- TODO: write one of these that traverses symbolic links but +-- efficiently ignores loops. findFiles :: Path Abs Dir -- ^ Root directory to begin with. -> (Path Abs File -> Bool) -- ^ Predicate to match files. -> (Path Abs Dir -> Bool) -- ^ Predicate for which directories to traverse. @@ -60,13 +69,18 @@ else Nothing) (listDir dir) (\ _ -> return ([], [])) + filteredFiles <- evaluate $ force (filter p files) + filteredDirs <- filterM (fmap not . isSymLink) dirs subResults <- - forM dirs + forM filteredDirs (\entry -> if traversep entry then findFiles entry p traversep else return []) - return (concat (filter p files : subResults)) + return (concat (filteredFiles : subResults)) + +isSymLink :: Path Abs t -> IO Bool +isSymLink = fmap isSymbolicLink . getSymbolicLinkStatus . toFilePath -- | @findInParents f path@ applies @f@ to @path@ and its 'parent's until -- it finds a 'Just' or reaches the root directory. diff --git a/src/Stack/Build.hs b/src/Stack/Build.hs --- a/src/Stack/Build.hs +++ b/src/Stack/Build.hs @@ -15,9 +15,12 @@ (build ,withLoadPackage ,mkBaseConfigOpts - ,queryBuildInfo) + ,queryBuildInfo + ,splitObjsWarning + ,CabalVersionException(..)) where +import Control.Exception (Exception) import Control.Monad import Control.Monad.Catch (MonadCatch, MonadMask) import Control.Monad.IO.Class @@ -27,7 +30,6 @@ import Data.Aeson (Value (Object, Array), (.=), object) import Data.Function import qualified Data.HashMap.Strict as HM -import Data.IORef.RunOnce (runOnce) import Data.List ((\\)) import Data.List.Extra (groupSort) import Data.List.NonEmpty (NonEmpty(..)) @@ -42,6 +44,7 @@ import Data.Text.Encoding (decodeUtf8) import qualified Data.Text.IO as TIO import Data.Text.Read (decimal) +import Data.Typeable (Typeable) import qualified Data.Vector as V import qualified Data.Yaml as Yaml import Network.HTTP.Client.Conduit (HasHttpManager) @@ -61,11 +64,11 @@ import System.FileLock (FileLock, unlockFile) #ifdef WINDOWS -import System.Win32.Console (setConsoleCP, setConsoleOutputCP, getConsoleCP, getConsoleOutputCP) +import System.Win32.Console (setConsoleCP, setConsoleOutputCP, getConsoleCP, getConsoleOutputCP) import qualified Control.Monad.Catch as Catch #endif -type M env m = (MonadIO m,MonadReader env m,HasHttpManager env,HasBuildConfig env,MonadLogger m,MonadBaseControl IO m,MonadCatch m,MonadMask m,HasLogLevel env,HasEnvConfig env,HasTerminal env) +type M env m = (MonadIO m,MonadReader env m,HasHttpManager env,HasBuildConfig env,MonadLogger m,MonadBaseControl IO m,MonadMask m,HasLogLevel env,HasEnvConfig env,HasTerminal env) -- | Build. -- @@ -75,12 +78,14 @@ build :: M env m => (Set (Path Abs File) -> IO ()) -- ^ callback after discovering all local files -> Maybe FileLock - -> BuildOpts + -> BuildOptsCLI -> m () -build setLocalFiles mbuildLk bopts = fixCodePage $ do +build setLocalFiles mbuildLk boptsCli = fixCodePage $ do + bopts <- asks (configBuild . getConfig) + let profiling = boptsLibProfile bopts || boptsExeProfile bopts menv <- getMinimalEnvOverride - (_, mbp, locals, extraToBuild, sourceMap) <- loadSourceMap NeedTargets bopts + (targets, mbp, locals, extraToBuild, sourceMap) <- loadSourceMap NeedTargets boptsCli -- Set local files, necessary for file watching stackYaml <- asks $ bcStackYaml . getBuildConfig @@ -96,7 +101,7 @@ , getInstalledHaddock = shouldHaddockDeps bopts } sourceMap - baseConfigOpts <- mkBaseConfigOpts bopts + baseConfigOpts <- mkBaseConfigOpts boptsCli plan <- withLoadPackage menv $ \loadPackage -> constructPlan mbp baseConfigOpts locals extraToBuild localDumpPkgs loadPackage sourceMap installedMap @@ -110,21 +115,22 @@ liftIO $ unlockFile lk _ -> return () + checkCabalVersion + warnAboutSplitObjs bopts warnIfExecutablesWithSameNameCouldBeOverwritten locals plan when (boptsPreFetch bopts) $ preFetch plan - if boptsDryrun bopts + if boptsCLIDryrun boptsCli then printPlan plan - else executePlan menv bopts baseConfigOpts locals + else executePlan menv boptsCli baseConfigOpts locals globalDumpPkgs snapshotDumpPkgs localDumpPkgs installedMap + targets plan - where - profiling = boptsLibProfile bopts || boptsExeProfile bopts -- | If all the tasks are local, they don't mutate anything outside of our local directory. allLocal :: Plan -> Bool @@ -134,6 +140,23 @@ Map.elems . planTasks +checkCabalVersion :: M env m => m () +checkCabalVersion = do + allowNewer <- asks (configAllowNewer . getConfig) + cabalVer <- asks (envConfigCabalVersion . getEnvConfig) + -- https://github.com/haskell/cabal/issues/2023 + when (allowNewer && cabalVer < $(mkVersion "1.22")) $ throwM $ + CabalVersionException $ + "Error: --allow-newer requires at least Cabal version 1.22, but version " ++ + versionString cabalVer ++ + " was found." + +data CabalVersionException = CabalVersionException { unCabalVersionException :: String } + deriving (Typeable) + +instance Show CabalVersionException where show = unCabalVersionException +instance Exception CabalVersionException + -- | See https://github.com/commercialhaskell/stack/issues/1198. warnIfExecutablesWithSameNameCouldBeOverwritten :: MonadLogger m => [LocalPackage] -> Plan -> m () @@ -207,10 +230,24 @@ collect :: Ord k => [(k,v)] -> Map k (NonEmpty v) collect = Map.map NE.fromList . Map.fromDistinctAscList . groupSort +warnAboutSplitObjs :: MonadLogger m => BuildOpts -> m () +warnAboutSplitObjs bopts | boptsSplitObjs bopts = do + $logWarn $ "Building with --split-objs is enabled. " <> T.pack splitObjsWarning +warnAboutSplitObjs _ = return () + +splitObjsWarning :: String +splitObjsWarning = unwords + [ "Note that this feature is EXPERIMENTAL, and its behavior may be changed and improved." + , "You will need to clean your workdirs before use. If you want to compile all dependencies" + , "with split-objs, you will need to delete the snapshot (and all snapshots that could" + , "reference that snapshot)." + ] + -- | Get the @BaseConfigOpts@ necessary for constructing configure options mkBaseConfigOpts :: (MonadIO m, MonadReader env m, HasEnvConfig env, MonadThrow m) - => BuildOpts -> m BaseConfigOpts -mkBaseConfigOpts bopts = do + => BuildOptsCLI -> m BaseConfigOpts +mkBaseConfigOpts boptsCli = do + bopts <- asks (configBuild . getConfig) snapDBPath <- packageDatabaseDeps localDBPath <- packageDatabaseLocal snapInstallRoot <- installationRootDeps @@ -222,6 +259,7 @@ , bcoSnapInstallRoot = snapInstallRoot , bcoLocalInstallRoot = localInstallRoot , bcoBuildOpts = bopts + , bcoBuildOptsCLI = boptsCli , bcoExtraDBs = packageExtraDBs } @@ -238,16 +276,15 @@ -> m a withLoadPackage menv inner = do econfig <- asks getEnvConfig - withCabalLoader' <- runOnce $ withCabalLoader menv $ \cabalLoader -> + withCabalLoader menv $ \cabalLoader -> inner $ \name version flags -> do - bs <- cabalLoader $ PackageIdentifier name version -- TODO automatically update index the first time this fails + bs <- cabalLoader $ PackageIdentifier name version -- Intentionally ignore warnings, as it's not really -- appropriate to print a bunch of warnings out while -- resolving the package index. (_warnings,pkg) <- readPackageBS (depPackageConfig econfig flags) bs return pkg - withCabalLoader' where -- | Package config to be used for dependencies depPackageConfig :: EnvConfig -> Map FlagName Bool -> PackageConfig @@ -337,7 +374,7 @@ -- | Get the raw build information object rawBuildInfo :: M env m => m Value rawBuildInfo = do - (_, _mbp, locals, _extraToBuild, _sourceMap) <- loadSourceMap NeedTargets defaultBuildOpts + (_, _mbp, locals, _extraToBuild, _sourceMap) <- loadSourceMap NeedTargets defaultBuildOptsCLI return $ object [ "locals" .= Object (HM.fromList $ map localToPair locals) ] diff --git a/src/Stack/Build/ConstructPlan.hs b/src/Stack/Build/ConstructPlan.hs --- a/src/Stack/Build/ConstructPlan.hs +++ b/src/Stack/Build/ConstructPlan.hs @@ -11,7 +11,7 @@ ( constructPlan ) where -import Control.Arrow ((&&&), second) +import Control.Arrow ((&&&)) import Control.Exception.Lifted import Control.Monad import Control.Monad.Catch (MonadCatch) @@ -32,18 +32,18 @@ import qualified Data.Text as T import Data.Text.Encoding (decodeUtf8With) import Data.Text.Encoding.Error (lenientDecode) -import Distribution.Package (Dependency (..)) -import Distribution.Version (anyVersion) +import qualified Distribution.Package as Cabal +import qualified Distribution.Version as Cabal import Network.HTTP.Client.Conduit (HasHttpManager) import Prelude hiding (pi, writeFile) import Stack.Build.Cache import Stack.Build.Haddock import Stack.Build.Installed import Stack.Build.Source -import Stack.Types.Build import Stack.BuildPlan import Stack.Package import Stack.PackageDump +import Stack.PackageIndex import Stack.Types data PackageInfo @@ -101,11 +101,11 @@ , baseConfigOpts :: !BaseConfigOpts , loadPackage :: !(PackageName -> Version -> Map FlagName Bool -> IO Package) , combinedMap :: !CombinedMap - , toolToPackages :: !(Dependency -> Map PackageName VersionRange) + , toolToPackages :: !(Cabal.Dependency -> Map PackageName VersionRange) , ctxEnvConfig :: !EnvConfig , callStack :: ![PackageName] , extraToBuild :: !(Set PackageName) - , ctxVersions :: !(Map PackageName (Set Version)) + , getVersions :: !(PackageName -> IO (Set Version)) , wanted :: !(Set PackageName) , localNames :: !(Set PackageName) } @@ -132,11 +132,7 @@ -> m Plan constructPlan mbp0 baseConfigOpts0 locals extraToBuild0 localDumpPkgs loadPackage0 sourceMap installedMap = do let locallyRegistered = Map.fromList $ map (dpGhcPkgId &&& dpPackageIdent) localDumpPkgs - bconfig <- asks getBuildConfig - let versions = - Map.fromListWith Set.union $ - map (second Set.singleton . toTuple) $ - Map.keys (bcPackageCaches bconfig) + getVersions0 <- getPackageVersionsIO econfig <- asks getEnvConfig let onWanted = void . addDep False . packageName . lpPackage @@ -144,7 +140,7 @@ mapM_ onWanted $ filter lpWanted locals mapM_ (addDep False) $ Set.toList extraToBuild0 ((), m, W efinals installExes dirtyReason deps warnings) <- - liftIO $ runRWST inner (ctx econfig versions) M.empty + liftIO $ runRWST inner (ctx econfig getVersions0) M.empty mapM_ $logWarn (warnings []) let toEither (_, Left e) = Left e toEither (k, Right v) = Right (k, v) @@ -157,7 +153,7 @@ toTask (name, ADRToInstall task) = Just (name, task) tasks = M.fromList $ mapMaybe toTask adrs takeSubset = - case boptsBuildSubset $ bcoBuildOpts baseConfigOpts0 of + case boptsCLIBuildSubset $ bcoBuildOptsCLI baseConfigOpts0 of BSAll -> id BSOnlySnapshot -> stripLocals BSOnlyDependencies -> stripNonDeps deps @@ -172,18 +168,18 @@ } else throwM $ ConstructPlanExceptions errs (bcStackYaml $ getBuildConfig econfig) where - ctx econfig versions = Ctx + ctx econfig getVersions0 = Ctx { mbp = mbp0 , baseConfigOpts = baseConfigOpts0 , loadPackage = loadPackage0 , combinedMap = combineMap sourceMap installedMap - , toolToPackages = \ (Dependency name _) -> - maybe Map.empty (Map.fromSet (const anyVersion)) $ + , toolToPackages = \(Cabal.Dependency name _) -> + maybe Map.empty (Map.fromSet (const Cabal.anyVersion)) $ Map.lookup (T.pack . packageNameString . fromCabalPackageName $ name) toolMap , ctxEnvConfig = econfig , callStack = [] , extraToBuild = extraToBuild0 - , ctxVersions = versions + , getVersions = getVersions0 , wanted = wantedLocalPackages locals , localNames = Set.fromList $ map (packageName . lpPackage) locals } @@ -430,15 +426,17 @@ deps' <- packageDepsWithTools package deps <- forM (Map.toList deps') $ \(depname, range) -> do eres <- addDep treatAsDep depname - let mlatestApplicable = - (latestApplicableVersion range <=< Map.lookup depname) (ctxVersions ctx) + let getLatestApplicable = do + vs <- liftIO $ getVersions ctx depname + return (latestApplicableVersion range vs) case eres of - Left e -> + Left e -> do let bd = case e of UnknownPackage name -> assert (name == depname) NotInBuildPlan _ -> Couldn'tResolveItsDependencies - in return $ Left (depname, (range, mlatestApplicable, bd)) + mlatestApplicable <- getLatestApplicable + return $ Left (depname, (range, mlatestApplicable, bd)) Right adr -> do inRange <- if adrVersion adr `withinRange` range then return True @@ -477,7 +475,9 @@ (Set.empty, Map.empty, loc) ADRFound loc (Library ident gid) -> return $ Right (Set.empty, Map.singleton ident gid, loc) - else return $ Left (depname, (range, mlatestApplicable, DependencyMismatch $ adrVersion adr)) + else do + mlatestApplicable <- getLatestApplicable + return $ Left (depname, (range, mlatestApplicable, DependencyMismatch $ adrVersion adr)) case partitionEithers deps of ([], pairs) -> return $ Right $ mconcat pairs (errs, _) -> return $ Left $ DependencyPlanFailures @@ -522,6 +522,7 @@ Nothing -> Just "old configure information not found" Just oldOpts | Just reason <- describeConfigDiff config oldOpts wantConfigCache -> Just reason + | True <- psForceDirty ps -> Just "--force-dirty specified" | Just files <- psDirty ps -> Just $ "local file changes: " <> addEllipsis (T.pack $ unwords $ Set.toList files) | otherwise -> Nothing @@ -547,25 +548,6 @@ ] | otherwise = Nothing where - -- options set by stack - isStackOpt t = any (`T.isPrefixOf` t) - [ "--dependency=" - , "--constraint=" - , "--package-db=" - , "--libdir=" - , "--bindir=" - , "--datadir=" - , "--libexecdir=" - , "--sysconfdir" - , "--docdir=" - , "--htmldir=" - , "--haddockdir=" - , "--enable-tests" - , "--enable-benchmarks" - ] || elem t - [ "--user" - ] - stripGhcOptions = go where @@ -606,6 +588,10 @@ newComponents = configCacheComponents new `Set.difference` configCacheComponents old +psForceDirty :: PackageSource -> Bool +psForceDirty (PSLocal lp) = lpForceDirty lp +psForceDirty (PSUpstream {}) = False + psDirty :: PackageSource -> Maybe (Set FilePath) psDirty (PSLocal lp) = lpDirtyFiles lp psDirty (PSUpstream {}) = Nothing -- files never change in an upstream package @@ -623,9 +609,51 @@ packageDepsWithTools :: Package -> M (Map PackageName VersionRange) packageDepsWithTools p = do ctx <- ask + -- TODO: it would be cool to defer these warnings until there's an + -- actual issue building the package. + -- TODO: check if the tool is on the path before warning? + let toEither (Cabal.Dependency (Cabal.PackageName name) _) mp = + case Map.toList mp of + [] -> Left (NoToolFound name (packageName p)) + [_] -> Right mp + xs -> Left (AmbiguousToolsFound name (packageName p) (map fst xs)) + (warnings, toolDeps) = + partitionEithers $ + map (\dep -> toEither dep (toolToPackages ctx dep)) (packageTools p) + tell mempty { wWarnings = (map toolWarningText warnings ++) } + when (any isNoToolFound warnings) $ do + let msg = T.unlines + [ "Missing build-tools may be caused by dependencies of the build-tool being overridden by extra-deps." + , "This should be fixed soon - see this issue https://github.com/commercialhaskell/stack/issues/595" + ] + tell mempty { wWarnings = (msg:) } return $ Map.unionsWith intersectVersionRanges $ packageDeps p - : map (toolToPackages ctx) (packageTools p) + : toolDeps + +data ToolWarning + = NoToolFound String PackageName + | AmbiguousToolsFound String PackageName [PackageName] + +isNoToolFound :: ToolWarning -> Bool +isNoToolFound NoToolFound{} = True +isNoToolFound _ = False + +toolWarningText :: ToolWarning -> Text +toolWarningText (NoToolFound toolName pkgName) = + "No packages found in snapshot which provide a " <> + T.pack (show toolName) <> + " executable, which is a build-tool dependency of " <> + T.pack (show (packageNameString pkgName)) +toolWarningText (AmbiguousToolsFound toolName pkgName options) = + "Multiple packages found in snapshot which provide a " <> + T.pack (show toolName) <> + " exeuctable, which is a build-tool dependency of " <> + T.pack (show (packageNameString pkgName)) <> + ", so none will be installed.\n" <> + "Here's the list of packages which provide it: " <> + T.intercalate ", " (map packageNameText options) <> + "\nSince there's no good way to choose, you may need to install it manually." -- | Strip out anything from the @Plan@ intended for the local database stripLocals :: Plan -> Plan diff --git a/src/Stack/Build/Execute.hs b/src/Stack/Build/Execute.hs --- a/src/Stack/Build/Execute.hs +++ b/src/Stack/Build/Execute.hs @@ -17,13 +17,13 @@ ) where import Control.Applicative -import Control.Arrow ((&&&)) +import Control.Arrow ((&&&), second) import Control.Concurrent.Execute import Control.Concurrent.MVar.Lifted import Control.Concurrent.STM import Control.Exception.Enclosed (catchIO) import Control.Exception.Lifted -import Control.Monad (liftM, when, unless, void, join) +import Control.Monad (liftM, when, unless, void) import Control.Monad.Catch (MonadCatch, MonadMask) import Control.Monad.Extra (anyM, (&&^)) import Control.Monad.IO.Class @@ -31,7 +31,7 @@ import Control.Monad.Reader (MonadReader, asks) import Control.Monad.Trans.Control (liftBaseWith) import Control.Monad.Trans.Resource -import Data.Attoparsec.Text +import Data.Attoparsec.Text hiding (try) import qualified Data.ByteString as S import Data.Char (isSpace) import Data.Conduit @@ -50,26 +50,27 @@ import Data.Monoid ((<>)) import Data.Set (Set) import qualified Data.Set as Set -import qualified Data.Streaming.Process as Process import Data.Streaming.Process hiding (callProcess, env) import Data.Text (Text) import qualified Data.Text as T import Data.Text.Encoding (decodeUtf8) import Data.Time.Clock (getCurrentTime) import Data.Traversable (forM) +import Data.Tuple import qualified Distribution.PackageDescription as C import Distribution.System (OS (Windows), Platform (Platform)) import Language.Haskell.TH as TH (location) import Network.HTTP.Client.Conduit (HasHttpManager) import Path -import Path.Extra (toFilePathNoTrailingSep) +import Path.Extra (toFilePathNoTrailingSep, rejectMissingFile) import Path.IO hiding (findExecutable, makeAbsolute) import Prelude hiding (FilePath, writeFile, any) import Stack.Build.Cache import Stack.Build.Haddock import Stack.Build.Installed import Stack.Build.Source +import Stack.Build.Target import Stack.Config import Stack.Constants import Stack.Coverage @@ -94,7 +95,7 @@ import System.Process.Internals (createProcess_) #endif -type M env m = (MonadIO m,MonadReader env m,HasHttpManager env,HasBuildConfig env,MonadLogger m,MonadBaseControl IO m,MonadCatch m,MonadMask m,HasLogLevel env,HasEnvConfig env,HasTerminal env, HasConfig env) +type M env m = (MonadIO m,MonadReader env m,HasHttpManager env,HasBuildConfig env,MonadLogger m,MonadBaseControl IO m,MonadMask m,HasLogLevel env,HasEnvConfig env,HasTerminal env, HasConfig env) -- | Fetch the packages necessary for a build, for example in combination with a dry run. preFetch :: M env m => Plan -> m () @@ -199,6 +200,7 @@ , eeConfigureLock :: !(MVar ()) , eeInstallLock :: !(MVar ()) , eeBuildOpts :: !BuildOpts + , eeBuildOptsCLI :: !BuildOptsCLI , eeBaseConfigOpts :: !BaseConfigOpts , eeGhcPkgIds :: !(TVar (Map PackageIdentifier Installed)) , eeTempDir :: !(Path Abs Dir) @@ -285,6 +287,7 @@ withExecuteEnv :: M env m => EnvOverride -> BuildOpts + -> BuildOptsCLI -> BaseConfigOpts -> [LocalPackage] -> [DumpPackage () ()] -- ^ global packages @@ -292,7 +295,7 @@ -> [DumpPackage () ()] -- ^ local packages -> (ExecuteEnv -> m a) -> m a -withExecuteEnv menv bopts baseConfigOpts locals globalPackages snapshotPackages localPackages inner = do +withExecuteEnv menv bopts boptsCli baseConfigOpts locals globalPackages snapshotPackages localPackages inner = do withSystemTempDir stackProgName $ \tmpdir -> do configLock <- newMVar () installLock <- newMVar () @@ -307,6 +310,7 @@ inner ExecuteEnv { eeEnvOverride = menv , eeBuildOpts = bopts + , eeBuildOptsCLI = boptsCli -- Uncertain as to why we cannot run configures in parallel. This appears -- to be a Cabal library bug. Original issue: -- https://github.com/fpco/stack/issues/84. Ideally we'd be able to remove @@ -333,17 +337,19 @@ -- | Perform the actual plan executePlan :: M env m => EnvOverride - -> BuildOpts + -> BuildOptsCLI -> BaseConfigOpts -> [LocalPackage] -> [DumpPackage () ()] -- ^ global packages -> [DumpPackage () ()] -- ^ snapshot packages -> [DumpPackage () ()] -- ^ local packages -> InstalledMap + -> Map PackageName SimpleTarget -> Plan -> m () -executePlan menv bopts baseConfigOpts locals globalPackages snapshotPackages localPackages installedMap plan = do - withExecuteEnv menv bopts baseConfigOpts locals globalPackages snapshotPackages localPackages (executePlan' installedMap plan) +executePlan menv boptsCli baseConfigOpts locals globalPackages snapshotPackages localPackages installedMap targets plan = do + bopts <- asks (configBuild . getConfig) + withExecuteEnv menv bopts boptsCli baseConfigOpts locals globalPackages snapshotPackages localPackages (executePlan' installedMap targets plan) unless (Map.null $ planInstallExes plan) $ do snapBin <- ( bindirSuffix) `liftM` installationRootDeps @@ -367,6 +373,7 @@ Snap -> snapBin Local -> localBin mfp <- forgivingAbsence (resolveFile bindir $ T.unpack name ++ ext) + >>= rejectMissingFile case mfp of Nothing -> do $logWarn $ T.concat @@ -446,7 +453,7 @@ , esStackExe = True , esLocaleUtf8 = False } - forM_ (boptsExec bopts) $ \(cmd, args) -> do + forM_ (boptsCLIExec boptsCli) $ \(cmd, args) -> do $logProcessRun cmd args callProcess (Cmd Nothing cmd menv' args) @@ -464,10 +471,11 @@ -- | Perform the actual plan (internal) executePlan' :: M env m => InstalledMap + -> Map PackageName SimpleTarget -> Plan -> ExecuteEnv -> m () -executePlan' installedMap0 plan ee@ExecuteEnv {..} = do +executePlan' installedMap0 targets plan ee@ExecuteEnv {..} = do when (toCoverage $ boptsTestOpts eeBuildOpts) deleteHpcReports wc <- getWhichCompiler cv <- asks $ envConfigCompilerVersion . getEnvConfig @@ -544,6 +552,16 @@ generateLocalHaddockIndex eeEnvOverride wc eeBaseConfigOpts localDumpPkgs eeLocals generateDepsHaddockIndex eeEnvOverride wc eeBaseConfigOpts eeGlobalDumpPkgs snapshotDumpPkgs localDumpPkgs eeLocals generateSnapHaddockIndex eeEnvOverride wc eeBaseConfigOpts eeGlobalDumpPkgs snapshotDumpPkgs + when (boptsOpenHaddocks eeBuildOpts) $ do + let planPkgs, localPkgs, installedPkgs, availablePkgs + :: Map PackageName (PackageIdentifier, InstallLocation) + planPkgs = Map.map (taskProvides &&& taskLocation) (planTasks plan) + localPkgs = + Map.fromList + [(packageName p, (packageIdentifier p, Local)) | p <- map lpPackage eeLocals] + installedPkgs = Map.map (swap . second installedPackageIdentifier) installedMap' + availablePkgs = Map.unions [planPkgs, localPkgs, installedPkgs] + openHaddocksInBrowser eeBaseConfigOpts availablePkgs (Map.keysSet targets) where installedMap' = Map.difference installedMap0 $ Map.fromList @@ -605,10 +623,11 @@ beopts = boptsBenchmarkOpts bopts -- | Generate the ConfigCache -getConfigCache :: MonadIO m +getConfigCache :: M env m => ExecuteEnv -> Task -> InstalledMap -> Bool -> Bool -> m (Map PackageIdentifier GhcPkgId, ConfigCache) getConfigCache ExecuteEnv {..} Task {..} installedMap enableTest enableBench = do + useExactConf <- asks (configAllowNewer . getConfig) let extra = -- We enable tests if the test suite dependencies are already -- installed, so that we avoid unnecessary recompilation based on @@ -617,8 +636,15 @@ -- https://github.com/commercialhaskell/stack/issues/805 case taskType of TTLocal lp -> concat - [ ["--enable-tests" | enableTest || (depsPresent installedMap $ lpTestDeps lp)] - , ["--enable-benchmarks" | enableBench || (depsPresent installedMap $ lpBenchDeps lp)] + -- FIXME: make this work with exact-configuration. + -- Not sure how to plumb the info atm. See + -- https://github.com/commercialhaskell/stack/issues/2049 + [ [ "--enable-tests" + | enableTest || + (not useExactConf && depsPresent installedMap (lpTestDeps lp))] + , [ "--enable-benchmarks" + | enableBench || + (not useExactConf && depsPresent installedMap (lpBenchDeps lp))] ] _ -> [] idMap <- liftIO $ readTVarIO eeGhcPkgIds @@ -689,6 +715,8 @@ return $ case mpath of Nothing -> [] Just x -> return $ concat ["--with-", name, "=", toFilePath x] + -- Configure cabal with arguments determined by + -- Stack.Types.Build.configureOpts cabal False $ "configure" : concat [ concat exes , dirs @@ -781,11 +809,8 @@ , esLocaleUtf8 = True } menv <- liftIO $ configEnvOverride config envSettings - -- When looking for ghc to build Setup.hs we want to ignore local binaries, see: - -- https://github.com/commercialhaskell/stack/issues/1052 - menvWithoutLocals <- liftIO $ configEnvOverride config envSettings { esIncludeLocals = False } - getGhcPath <- runOnce $ liftIO $ join $ findExecutable menvWithoutLocals "ghc" - getGhcjsPath <- runOnce $ liftIO $ join $ findExecutable menvWithoutLocals "ghcjs" + getGhcPath <- runOnce $ getCompilerPath Ghc + getGhcjsPath <- runOnce $ getCompilerPath Ghcjs distRelativeDir' <- distRelativeDir esetupexehs <- -- Avoid broken Setup.hs files causing problems for simple build @@ -856,7 +881,7 @@ liftIO $ hClose h runResourceT $ CB.sourceFile (toFilePath logFile) - =$= CT.decodeUtf8 + =$= CT.decodeUtf8Lenient $$ mungeBuildOutput stripTHLoading makeAbsolute pkgDir =$ CL.consume throwM $ CabalExitedUnsuccessfully @@ -875,7 +900,7 @@ (outputSink False LevelWarn) (outputSink stripTHLoading LevelInfo) outputSink excludeTH level = - CT.decodeUtf8 + CT.decodeUtf8Lenient =$ mungeBuildOutput excludeTH makeAbsolute pkgDir =$ CL.mapM_ (runInBase . monadLoggerLog $(TH.location >>= liftLoc) "" level) -- If users want control, we should add a config option for this @@ -1041,7 +1066,7 @@ $ \package cabalfp pkgDir cabal announce _console _mlogFile -> do _neededConfig <- ensureConfig cache pkgDir ee (announce ("configure" <> annSuffix)) cabal cabalfp - if boptsOnlyConfigure eeBuildOpts + if boptsCLIOnlyConfigure eeBuildOptsCLI then return Nothing else liftM Just $ realBuild cache package pkgDir cabal announce @@ -1088,7 +1113,11 @@ unless isFinalBuild $ withMVar eeInstallLock $ \() -> do announce "copy/register" - cabal False ["copy"] + eres <- try $ cabal False ["copy"] + case eres of + Left err@CabalExitedUnsuccessfully{} -> + throwM $ CabalCopyFailed (packageSimpleType package) (show err) + _ -> return () when (packageHasLibrary package) $ cabal False ["register"] let (installedPkgDb, installedDumpPkgsTVar) = @@ -1243,16 +1272,12 @@ case mlogFile of Nothing -> Inherit Just (_, h) -> UseHandle h - cp = (proc (toFilePath exePath) args) - { cwd = Just $ toFilePath pkgDir - , Process.env = envHelper menv - , std_in = CreatePipe - , std_out = output - , std_err = output - } -- Use createProcess_ to avoid the log file being closed afterwards - (Just inH, Nothing, Nothing, ph) <- liftIO $ createProcess_ "singleBuild.runTests" cp + (Just inH, Nothing, Nothing, ph) <- createProcess' + stestName + (\cp -> cp { std_in = CreatePipe, std_out = output, std_err = output }) + (Cmd (Just pkgDir) (toFilePath exePath) menv args) when isTestTypeLib $ do logPath <- buildLogPath package (Just stestName) ensureDir (parent logPath) @@ -1271,12 +1296,11 @@ ExitSuccess -> Map.empty _ -> Map.singleton testName $ Just ec else do - $logError $ T.concat - [ "Test suite " - , testName - , " executable not found for " - , packageNameText $ packageName package - ] + $logError $ T.pack $ show $ TestSuiteExeMissing + (packageSimpleType package) + exeName + (packageNameString (packageName package)) + (T.unpack testName) return $ Map.singleton testName Nothing when needHpc $ do @@ -1310,8 +1334,6 @@ -> InstalledMap -> m () singleBench runInBase beopts benchesToRun ac ee task installedMap = do - -- FIXME: Since this doesn't use cabal, we should be able to avoid using a - -- fullblown 'withSingleContext'. (allDepsMap, _cache) <- getConfigCache ee task installedMap False True withSingleContext runInBase ac ee task (Just allDepsMap) (Just "bench") $ \_package _cabalfp _pkgDir cabal announce _console _mlogFile -> do let args = map T.unpack benchesToRun <> maybe [] diff --git a/src/Stack/Build/Haddock.hs b/src/Stack/Build/Haddock.hs --- a/src/Stack/Build/Haddock.hs +++ b/src/Stack/Build/Haddock.hs @@ -11,41 +11,88 @@ ( generateLocalHaddockIndex , generateDepsHaddockIndex , generateSnapHaddockIndex + , openHaddocksInBrowser , shouldHaddockPackage , shouldHaddockDeps ) where -import Control.Exception (tryJust, onException) +import Control.Exception (tryJust, onException) import Control.Monad -import Control.Monad.Catch (MonadCatch) +import Control.Monad.Catch (MonadCatch) import Control.Monad.IO.Class import Control.Monad.Logger +import Control.Monad.Reader import Control.Monad.Trans.Resource -import qualified Data.Foldable as F +import qualified Data.Foldable as F import Data.Function -import qualified Data.HashSet as HS +import qualified Data.HashSet as HS import Data.List -import Data.List.Extra (nubOrd) -import Data.Map.Strict (Map) -import qualified Data.Map.Strict as Map +import Data.List.Extra (nubOrd) +import Data.Map.Strict (Map) +import qualified Data.Map.Strict as Map import Data.Maybe -import Data.Maybe.Extra (mapMaybeM) -import Data.Set (Set) -import qualified Data.Set as Set -import Data.Text (Text) -import qualified Data.Text as T -import Data.Time (UTCTime) +import Data.Maybe.Extra (mapMaybeM) +import Data.Monoid ((<>)) +import Data.Set (Set) +import qualified Data.Set as Set +import Data.Text (Text) +import qualified Data.Text as T +import Data.Time (UTCTime) import Path import Path.Extra import Path.IO import Prelude -import Stack.Types.Build import Stack.PackageDump import Stack.Types -import qualified System.FilePath as FP -import System.IO.Error (isDoesNotExistError) +import qualified System.FilePath as FP +import System.IO.Error (isDoesNotExistError) import System.Process.Read +import Web.Browser (openBrowser) +openHaddocksInBrowser + :: (MonadIO m, MonadReader env m, HasBuildConfig env, MonadThrow m, MonadLogger m) + => BaseConfigOpts + -> Map PackageName (PackageIdentifier, InstallLocation) + -- ^ Available packages and their locations for the current project + -> Set PackageName + -- ^ Build targets as determined by 'Stack.Build.Source.loadSourceMap' + -> m () +openHaddocksInBrowser bco pkgLocations buildTargets = do + let cliTargets = (boptsCLITargets . bcoBuildOptsCLI) bco + getDocIndex = do + let localDocs = haddockIndexFile (localDepsDocDir bco) + localExists <- doesFileExist localDocs + if localExists + then return localDocs + else do + let snapDocs = haddockIndexFile (snapDocDir bco) + snapExists <- doesFileExist snapDocs + if snapExists + then return snapDocs + else fail "No local or snapshot doc index found to open." + docFile <- + case (cliTargets, map (`Map.lookup` pkgLocations) (Set.toList buildTargets)) of + ([_], [Just (pkgId, iloc)]) -> do + pkgRelDir <- (parseRelDir . packageIdentifierString) pkgId + let docLocation = + case iloc of + Snap -> snapDocDir bco + Local -> localDocDir bco + let docFile = haddockIndexFile (docLocation pkgRelDir) + exists <- doesFileExist docFile + if exists + then return docFile + else do + $logWarn $ + "Expected to find documentation at " <> + T.pack (toFilePath docFile) <> + ", but that file is missing. Opening doc index instead." + getDocIndex + _ -> getDocIndex + $logInfo ("Opening " <> T.pack (toFilePath docFile) <> " in the browser.") + _ <- liftIO $ openBrowser (toFilePath docFile) + return () + -- | Determine whether we should haddock for a package. shouldHaddockPackage :: BuildOpts -> Set PackageName -- ^ Packages that we want to generate haddocks for @@ -100,7 +147,7 @@ -> m () generateDepsHaddockIndex envOverride wc bco globalDumpPkgs snapshotDumpPkgs localDumpPkgs locals = do let deps = (mapMaybe (`lookupDumpPackage` allDumpPkgs) . nubOrd . findTransitiveDepends . mapMaybe getGhcPkgId) locals - depDocDir = localDocDir bco $(mkRelDir "all") + depDocDir = localDepsDocDir bco generateHaddockIndex "local packages and dependencies" envOverride @@ -228,7 +275,7 @@ ignoringAbsence (removeDirRecur destHtmlAbsDir) ensureDir destHtmlAbsDir onException - (copyDirRecur (parent srcInterfaceAbsFile) destHtmlAbsDir) + (copyDirRecur' (parent srcInterfaceAbsFile) destHtmlAbsDir) (ignoringAbsence (removeDirRecur destHtmlAbsDir)) destHtmlAbsDir = parent destInterfaceAbsFile @@ -246,6 +293,10 @@ -- | Path of local packages documentation directory. localDocDir :: BaseConfigOpts -> Path Abs Dir localDocDir bco = bcoLocalInstallRoot bco docDirSuffix + +-- | Path of documentation directory for the dependencies of local packages +localDepsDocDir :: BaseConfigOpts -> Path Abs Dir +localDepsDocDir bco = localDocDir bco $(mkRelDir "all") -- | Path of snapshot packages documentation directory. snapDocDir :: BaseConfigOpts -> Path Abs Dir diff --git a/src/Stack/Build/Installed.hs b/src/Stack/Build/Installed.hs --- a/src/Stack/Build/Installed.hs +++ b/src/Stack/Build/Installed.hs @@ -13,7 +13,7 @@ import Control.Applicative import Control.Monad -import Control.Monad.Catch (MonadCatch, MonadMask) +import Control.Monad.Catch (MonadMask) import Control.Monad.IO.Class import Control.Monad.Logger import Control.Monad.Reader (MonadReader, asks) @@ -43,7 +43,7 @@ import Stack.Types import Stack.Types.Internal -type M env m = (MonadIO m,MonadReader env m,HasHttpManager env,HasEnvConfig env,MonadLogger m,MonadBaseControl IO m,MonadCatch m,MonadMask m,HasLogLevel env) +type M env m = (MonadIO m,MonadReader env m,HasHttpManager env,HasEnvConfig env,MonadLogger m,MonadBaseControl IO m,MonadMask m,HasLogLevel env) -- | Options for 'getInstalled'. data GetInstalledOpts = GetInstalledOpts diff --git a/src/Stack/Build/Source.hs b/src/Stack/Build/Source.hs --- a/src/Stack/Build/Source.hs +++ b/src/Stack/Build/Source.hs @@ -61,6 +61,7 @@ parseCustomMiniBuildPlan) import Stack.Constants (wiredInPackages) import Stack.Package +import Stack.PackageIndex (getPackageVersions) import Stack.Types import qualified System.Directory as D @@ -69,32 +70,26 @@ loadSourceMap :: (MonadIO m, MonadCatch m, MonadReader env m, HasBuildConfig env, MonadBaseControl IO m, HasHttpManager env, MonadLogger m, HasEnvConfig env) => NeedTargets - -> BuildOpts + -> BuildOptsCLI -> m ( Map PackageName SimpleTarget , MiniBuildPlan , [LocalPackage] , Set PackageName -- non-local targets , SourceMap ) -loadSourceMap needTargets bopts = do +loadSourceMap needTargets boptsCli = do bconfig <- asks getBuildConfig rawLocals <- getLocalPackageViews - (mbp0, cliExtraDeps, targets) <- parseTargetsFromBuildOpts needTargets bopts - let latestVersion = - Map.fromListWith max $ - map toTuple $ - Map.keys (bcPackageCaches bconfig) - + (mbp0, cliExtraDeps, targets) <- parseTargetsFromBuildOpts needTargets boptsCli -- Extend extra-deps to encompass targets requested on the command line -- that are not in the snapshot. extraDeps0 <- extendExtraDeps (bcExtraDeps bconfig) cliExtraDeps (Map.keysSet $ Map.filter (== STUnknown) targets) - latestVersion - locals <- mapM (loadLocalPackage bopts targets) $ Map.toList rawLocals - checkFlagsUsed bopts locals extraDeps0 (mbpPackages mbp0) + locals <- mapM (loadLocalPackage boptsCli targets) $ Map.toList rawLocals + checkFlagsUsed boptsCli locals extraDeps0 (mbpPackages mbp0) checkComponentsBuildable locals let @@ -122,8 +117,8 @@ -- Overwrite any flag settings with those from the config file extraDeps3 = Map.mapWithKey (\n (v, f) -> PSUpstream v Local $ - case ( Map.lookup (Just n) $ boptsFlags bopts - , Map.lookup Nothing $ boptsFlags bopts + case ( Map.lookup (Just n) $ boptsCLIFlags boptsCli + , Map.lookup Nothing $ boptsCLIFlags boptsCli , Map.lookup n $ bcFlags bconfig ) of -- Didn't have any flag overrides, fall back to the flags @@ -154,9 +149,9 @@ parseTargetsFromBuildOpts :: (MonadIO m, MonadCatch m, MonadReader env m, HasBuildConfig env, MonadBaseControl IO m, HasHttpManager env, MonadLogger m, HasEnvConfig env) => NeedTargets - -> BuildOpts + -> BuildOptsCLI -> m (MiniBuildPlan, M.Map PackageName Version, M.Map PackageName SimpleTarget) -parseTargetsFromBuildOpts needTargets bopts = do +parseTargetsFromBuildOpts needTargets boptscli = do bconfig <- asks getBuildConfig mbp0 <- case bcResolver bconfig of @@ -183,7 +178,7 @@ snapshot (bcExtraDeps bconfig) rawLocals - (catMaybes $ Map.keys $ boptsFlags bopts) + (catMaybes $ Map.keys $ boptsCLIFlags boptscli) (cliExtraDeps, targets) <- parseTargets @@ -193,7 +188,7 @@ (flagExtraDeps <> bcExtraDeps bconfig) (fst <$> rawLocals) workingDir - (boptsTargets bopts) + (boptsCLITargets boptscli) return (mbp0, cliExtraDeps <> flagExtraDeps, targets) -- | For every package in the snapshot which is referenced by a flag, give the @@ -226,7 +221,7 @@ => m (Map PackageName (LocalPackageView, GenericPackageDescription)) getLocalPackageViews = do econfig <- asks getEnvConfig - locals <- forM (Map.toList $ envConfigPackages econfig) $ \(dir, validWanted) -> do + locals <- forM (Map.toList $ envConfigPackages econfig) $ \(dir, treatLikeExtraDep) -> do cabalfp <- findOrGenerateCabalFile dir (warnings,gpkg) <- readPackageUnresolved cabalfp mapM_ (printCabalFileWarning cabalfp) warnings @@ -237,7 +232,7 @@ { lpvVersion = fromCabalVersion $ pkgVersion cabalID , lpvRoot = dir , lpvCabalFP = cabalfp - , lpvExtraDep = not validWanted + , lpvExtraDep = treatLikeExtraDep , lpvComponents = getNamedComponents gpkg } return (name, (lpv, gpkg)) @@ -281,13 +276,13 @@ loadLocalPackage :: forall m env. (MonadReader env m, HasEnvConfig env, MonadCatch m, MonadLogger m, MonadIO m) - => BuildOpts + => BuildOptsCLI -> Map PackageName SimpleTarget -> (PackageName, (LocalPackageView, GenericPackageDescription)) -> m LocalPackage -loadLocalPackage bopts targets (name, (lpv, gpkg)) = do - config <- getPackageConfig bopts name - +loadLocalPackage boptsCli targets (name, (lpv, gpkg)) = do + config <- getPackageConfig boptsCli name + bopts <- asks (configBuild . getConfig) let pkg = resolvePackage config gpkg mtarget = Map.lookup name targets @@ -347,8 +342,9 @@ , lpBenchDeps = packageDeps benchpkg , lpTestBench = btpkg , lpFiles = files + , lpForceDirty = boptsForceDirty bopts , lpDirtyFiles = - if not (Set.null dirtyFiles) || boptsForceDirty bopts + if not (Set.null dirtyFiles) then let tryStripPrefix y = fromMaybe y (stripPrefix (toFilePath $ lpvRoot lpv) y) in Just $ Set.map tryStripPrefix dirtyFiles @@ -373,17 +369,17 @@ -- | Ensure that the flags specified in the stack.yaml file and on the command -- line are used. checkFlagsUsed :: (MonadThrow m, MonadReader env m, HasBuildConfig env) - => BuildOpts + => BuildOptsCLI -> [LocalPackage] -> Map PackageName extraDeps -- ^ extra deps -> Map PackageName snapshot -- ^ snapshot, for error messages -> m () -checkFlagsUsed bopts lps extraDeps snapshot = do +checkFlagsUsed boptsCli lps extraDeps snapshot = do bconfig <- asks getBuildConfig -- Check if flags specified in stack.yaml and the command line are -- used, see https://github.com/commercialhaskell/stack/issues/617 - let flags = map (, FSCommandLine) [(k, v) | (Just k, v) <- Map.toList $ boptsFlags bopts] + let flags = map (, FSCommandLine) [(k, v) | (Just k, v) <- Map.toList $ boptsCLIFlags boptsCli] ++ map (, FSStackYaml) (Map.toList $ bcFlags bconfig) localNameMap = Map.fromList $ map (packageName . lpPackage &&& lpPackage) lps @@ -431,31 +427,35 @@ -- Originally part of https://github.com/commercialhaskell/stack/issues/272, -- this was then superseded by -- https://github.com/commercialhaskell/stack/issues/651 -extendExtraDeps :: (MonadThrow m, MonadReader env m, HasBuildConfig env) - => Map PackageName Version -- ^ original extra deps - -> Map PackageName Version -- ^ package identifiers from the command line - -> Set PackageName -- ^ all packages added on the command line - -> Map PackageName Version -- ^ latest versions in indices - -> m (Map PackageName Version) -- ^ new extradeps -extendExtraDeps extraDeps0 cliExtraDeps unknowns latestVersion - | null errs = return $ Map.unions $ extraDeps1 : unknowns' - | otherwise = do - bconfig <- asks getBuildConfig - throwM $ UnknownTargets - (Set.fromList errs) - Map.empty -- TODO check the cliExtraDeps for presence in index - (bcStackYaml bconfig) +extendExtraDeps + :: (HasBuildConfig env, MonadIO m, MonadLogger m, MonadReader env m, HasHttpManager env, MonadBaseControl IO m, MonadCatch m) + => Map PackageName Version -- ^ original extra deps + -> Map PackageName Version -- ^ package identifiers from the command line + -> Set PackageName -- ^ all packages added on the command line + -> m (Map PackageName Version) -- ^ new extradeps +extendExtraDeps extraDeps0 cliExtraDeps unknowns = do + (errs, unknowns') <- fmap partitionEithers $ mapM addUnknown $ Set.toList unknowns + case errs of + [] -> return $ Map.unions $ extraDeps1 : unknowns' + _ -> do + bconfig <- asks getBuildConfig + throwM $ UnknownTargets + (Set.fromList errs) + Map.empty -- TODO check the cliExtraDeps for presence in index + (bcStackYaml bconfig) where extraDeps1 = Map.union extraDeps0 cliExtraDeps - - (errs, unknowns') = partitionEithers $ map addUnknown $ Set.toList unknowns - addUnknown pn = + addUnknown pn = do case Map.lookup pn extraDeps1 of - Just _ -> Right Map.empty - Nothing -> - case Map.lookup pn latestVersion of - Just v -> Right $ Map.singleton pn v - Nothing -> Left pn + Just _ -> return (Right Map.empty) + Nothing -> do + mlatestVersion <- getLatestVersion pn + case mlatestVersion of + Just v -> return (Right $ Map.singleton pn v) + Nothing -> return (Left pn) + getLatestVersion pn = do + vs <- getPackageVersions pn + return (fmap fst (Set.maxView vs)) -- | Compare the current filesystem state to the cached information, and -- determine (1) if the files are dirty, and (2) the new cache values. @@ -566,16 +566,16 @@ -- | Get 'PackageConfig' for package given its name. getPackageConfig :: (MonadIO m, MonadThrow m, MonadCatch m, MonadLogger m, MonadReader env m, HasEnvConfig env) - => BuildOpts + => BuildOptsCLI -> PackageName -> m PackageConfig -getPackageConfig bopts name = do +getPackageConfig boptsCli name = do econfig <- asks getEnvConfig bconfig <- asks getBuildConfig return PackageConfig { packageConfigEnableTests = False , packageConfigEnableBenchmarks = False - , packageConfigFlags = localFlags (boptsFlags bopts) bconfig name + , packageConfigFlags = localFlags (boptsCLIFlags boptsCli) bconfig name , packageConfigCompilerVersion = envConfigCompilerVersion econfig , packageConfigPlatform = configPlatform $ getConfig bconfig } diff --git a/src/Stack/Build/Target.hs b/src/Stack/Build/Target.hs --- a/src/Stack/Build/Target.hs +++ b/src/Stack/Build/Target.hs @@ -20,21 +20,25 @@ ) where import Control.Applicative -import Control.Arrow (second) -import Control.Monad.Catch (MonadCatch, throwM) +import Control.Arrow (second) +import Control.Monad.Catch (MonadCatch, throwM) import Control.Monad.IO.Class -import Data.Either (partitionEithers) -import Data.Map (Map) -import qualified Data.Map as Map -import Data.Maybe (mapMaybe) -import Data.Monoid -import Data.Set (Set) -import qualified Data.Set as Set -import Data.Text (Text) -import qualified Data.Text as T +import Data.Either (partitionEithers) +import Data.Foldable +import Data.List.Extra (groupSort) +import Data.List.NonEmpty (NonEmpty((:|))) +import qualified Data.List.NonEmpty as NonEmpty +import Data.Map (Map) +import qualified Data.Map as Map +import Data.Maybe (mapMaybe) +import Data.Set (Set) +import qualified Data.Set as Set +import Data.Text (Text) +import qualified Data.Text as T import Path +import Path.Extra (rejectMissingDir) import Path.IO -import Prelude -- Fix redundant import warnings +import Prelude hiding (concat, concatMap) -- Fix redundant import warnings import Stack.Types -- | The name of a component, which applies to executables, test suites, and benchmarks @@ -101,7 +105,7 @@ , lpvRoot :: !(Path Abs Dir) , lpvCabalFP :: !(Path Abs File) , lpvComponents :: !(Set NamedComponent) - , lpvExtraDep :: !Bool + , lpvExtraDep :: !TreatLikeExtraDep } -- | Same as @parseRawTarget@, but also takes directories into account. @@ -115,6 +119,7 @@ Just rt -> return $ Right [(ri, rt)] Nothing -> do mdir <- forgivingAbsence (resolveDir root (T.unpack t)) + >>= rejectMissingDir case mdir of Nothing -> return $ Left $ "Directory not found: " `T.append` t Just dir -> @@ -147,31 +152,24 @@ resolveIdents _ _ _ (ri, RTComponent x) = Right ((ri, RTComponent x), Map.empty) resolveIdents _ _ _ (ri, RTPackage x) = Right ((ri, RTPackage x), Map.empty) resolveIdents snap extras locals (ri, RTPackageIdentifier (PackageIdentifier name version)) = - case mfound of - Just (foundPlace, foundVersion) | foundVersion /= version -> Left $ T.pack $ concat - [ "Specified target version " - , versionString version - , " for package " - , packageNameString name - , " does not match " - , foundPlace - , " version " - , versionString foundVersion - ] - _ -> Right - ( (ri, RTPackage name) - , case mfound of - -- Add to extra deps since we didn't have it already - Nothing -> Map.singleton name version - -- Already had it, don't add to extra deps - Just _ -> Map.empty - ) + fmap ((ri, RTPackage name), ) newExtras where - mfound = mlocal <|> mextra <|> msnap - - mlocal = (("local", ) . lpvVersion) <$> Map.lookup name locals - mextra = ("extra-deps", ) <$> Map.lookup name extras - msnap = ("snapshot", ) <$> Map.lookup name snap + newExtras = + case (Map.lookup name locals, mfound) of + -- Error if it matches a local package, pkg idents not + -- supported for local. + (Just _, _) -> Left $ T.concat + [ packageNameText name + , " target has a specific version number, but it is a local package." + , "\nTo avoid confusion, we will not install the specified version or build the local one." + , "\nTo build the local package, specify the target without an explicit version." + ] + -- If the found version matches, no need for an extra-dep. + (_, Just foundVersion) | foundVersion == version -> Right Map.empty + -- Otherwise, if there is no specified version or a + -- mismatch, add an extra-dep. + _ -> Right $ Map.singleton name version + mfound = asum (map (Map.lookup name) [extras, snap]) resolveRawTarget :: Map PackageName Version -- ^ snapshot -> Map PackageName Version -- ^ extra deps @@ -247,24 +245,26 @@ simplifyTargets :: [(PackageName, (RawInput, SimpleTarget))] -> ([Text], Map PackageName SimpleTarget) simplifyTargets = - mconcat . map go . Map.toList . Map.fromListWith (++) . fmap (second return) + foldMap go . collect where - go :: (PackageName, [(RawInput, SimpleTarget)]) + go :: (PackageName, NonEmpty (RawInput, SimpleTarget)) -> ([Text], Map PackageName SimpleTarget) - go (_, []) = error "Stack.Build.Target.simplifyTargets: the impossible happened" - go (name, [(_, st)]) = ([], Map.singleton name st) + go (name, (_, st) :| []) = ([], Map.singleton name st) go (name, pairs) = - case partitionEithers $ map (getLocalComp . snd) pairs of + case partitionEithers $ map (getLocalComp . snd) (NonEmpty.toList pairs) of ([], comps) -> ([], Map.singleton name $ STLocalComps $ Set.unions comps) _ -> let err = T.pack $ concat [ "Overlapping targets provided for package " , packageNameString name , ": " - , show $ map (unRawInput . fst) pairs + , show $ map (unRawInput . fst) (NonEmpty.toList pairs) ] in ([err], Map.empty) + collect :: Ord a => [(a, b)] -> [(a, NonEmpty b)] + collect = map (second NonEmpty.fromList) . groupSort + getLocalComp (STLocalComps comps) = Right comps getLocalComp _ = Left () @@ -283,9 +283,10 @@ -> [Text] -- ^ command line targets -> m (Map PackageName Version, Map PackageName SimpleTarget) parseTargets needTargets implicitGlobal snap extras locals currDir textTargets' = do - let textTargets = + let nonExtraDeps = Map.keys $ Map.filter (not . lpvExtraDep) locals + textTargets = if null textTargets' - then map (T.pack . packageNameString) $ Map.keys $ Map.filter (not . lpvExtraDep) locals + then map (T.pack . packageNameString) nonExtraDeps else textTargets' erawTargets <- mapM (parseRawTargetDirs currDir locals) textTargets @@ -302,10 +303,12 @@ then case needTargets of AllowNoTargets -> return (Map.empty, Map.empty) - NeedTargets -> - throwM $ TargetParseException - $ if implicitGlobal - then ["The specified targets matched no packages.\nPerhaps you need to run 'stack init'?"] - else ["The specified targets matched no packages"] + NeedTargets + | null textTargets' && implicitGlobal -> throwM $ TargetParseException + ["The specified targets matched no packages.\nPerhaps you need to run 'stack init'?"] + | null textTargets' && null nonExtraDeps -> throwM $ TargetParseException + ["The project contains no local packages (packages not marked with 'extra-dep')"] + | otherwise -> throwM $ TargetParseException + ["The specified targets matched no packages"] else return (Map.unions newExtras, targets) else throwM $ TargetParseException errs diff --git a/src/Stack/BuildPlan.hs b/src/Stack/BuildPlan.hs --- a/src/Stack/BuildPlan.hs +++ b/src/Stack/BuildPlan.hs @@ -24,10 +24,10 @@ , removeSrcPkgDefaultFlags , resolveBuildPlan , selectBestSnapshot - , ToolMap , getToolMap , shadowMiniBuildPlan , showItems + , showPackageFlags , parseCustomMiniBuildPlan ) where @@ -44,17 +44,18 @@ import qualified Crypto.Hash.SHA256 as SHA256 import Data.Aeson.Extended (FromJSON (..), withObject, (.:), (.:?), (.!=)) import Data.Binary.VersionTagged (taggedDecodeOrLoad) -import Data.ByteString (ByteString) -import qualified Data.ByteString.Base16 as B16 import qualified Data.ByteString as S +import qualified Data.ByteString.Base16 as B16 import qualified Data.ByteString.Char8 as S8 import Data.Either (partitionEithers) import qualified Data.Foldable as F import qualified Data.HashSet as HashSet import Data.List (intercalate) +import Data.List.NonEmpty (NonEmpty(..)) +import qualified Data.List.NonEmpty as NonEmpty import Data.Map (Map) import qualified Data.Map as Map -import Data.Maybe (fromJust, fromMaybe, mapMaybe) +import Data.Maybe (fromMaybe, mapMaybe) import Data.Monoid import Data.Set (Set) import qualified Data.Set as Set @@ -64,28 +65,29 @@ import qualified Data.Traversable as Tr import Data.Typeable (Typeable) import Data.Yaml (decodeEither', decodeFileEither) +import qualified Distribution.Package as C import Distribution.PackageDescription (GenericPackageDescription, flagDefault, flagManual, flagName, genPackageFlags, executables, exeName, library, libBuildInfo, buildable) -import Distribution.System (Platform) -import qualified Distribution.Package as C import qualified Distribution.PackageDescription as C -import qualified Distribution.Version as C +import Distribution.System (Platform) import Distribution.Text (display) +import qualified Distribution.Version as C +import Network.HTTP.Client (checkStatus) import Network.HTTP.Download import Network.HTTP.Types (Status(..)) -import Network.HTTP.Client (checkStatus) import Path import Path.IO import Prelude -- Fix AMP warning import Stack.Constants import Stack.Fetch import Stack.Package +import Stack.PackageIndex import Stack.Types import Stack.Types.StackT import qualified System.Directory as D -import qualified System.FilePath as FP +import qualified System.FilePath as FP data BuildPlanException = UnknownPackages @@ -190,10 +192,11 @@ | Map.null (rsUnknown rs) && Map.null (rsShadowed rs) = return (rsToInstall rs, rsUsedBy rs) | otherwise = do bconfig <- asks getBuildConfig + caches <- getPackageCaches let maxVer = Map.fromListWith max $ map toTuple $ - Map.keys (bcPackageCaches bconfig) + Map.keys caches unknown = flip Map.mapWithKey (rsUnknown rs) $ \ident x -> (Map.lookup ident maxVer, x) throwM $ UnknownPackages @@ -264,7 +267,7 @@ if allowMissing then do (missingNames, missingIdents, m) <- - resolvePackagesAllowMissing menv (Map.keysSet idents0) Set.empty + resolvePackagesAllowMissing (Map.keysSet idents0) Set.empty assert (Set.null missingNames) $ return (m, missingIdents) else do @@ -308,7 +311,7 @@ $ map (\(n, (v, f)) -> (PackageIdentifier n v, Left f)) $ Map.toList toCalc --- | Resolve all packages necessary to install for +-- | Resolve all packages necessary to install for the needed packages. getDeps :: MiniBuildPlan -> (PackageName -> Bool) -- ^ is it shadowed by a local package? -> Map PackageName (Set PackageName) @@ -370,9 +373,6 @@ } return shadowed --- | Look up with packages provide which tools. -type ToolMap = Map ByteString (Set PackageName) - -- | Map from tool name to package providing it getToolMap :: MiniBuildPlan -> Map Text (Set PackageName) getToolMap mbp = @@ -434,7 +434,7 @@ -- | Load the 'BuildPlan' for the given snapshot. Will load from a local copy -- if available, otherwise downloading from Github. -loadBuildPlan :: (MonadIO m, MonadThrow m, MonadLogger m, MonadReader env m, HasHttpManager env, HasStackRoot env) +loadBuildPlan :: (MonadIO m, MonadThrow m, MonadLogger m, MonadReader env m, HasHttpManager env, HasStackRoot env, HasConfig env) => SnapName -> m BuildPlan loadBuildPlan name = do @@ -449,6 +449,7 @@ Left e -> do $logDebug $ "Decoding build plan from file failed: " <> T.pack (show e) ensureDir (parent fp) + url <- buildBuildPlanUrl name file req <- parseUrl $ T.unpack url $logSticky $ "Downloading " <> renderSnapName name <> " build plan ..." $logDebug $ "Downloading build plan from: " <> url @@ -458,14 +459,17 @@ where file = renderSnapName name <> ".yaml" - reponame = - case name of - LTS _ _ -> "lts-haskell" - Nightly _ -> "stackage-nightly" - url = rawGithubUrl "fpco" reponame "master" file handle404 (Status 404 _) _ _ = Just $ SomeException $ SnapshotNotFound name handle404 _ _ _ = Nothing +buildBuildPlanUrl :: (MonadReader env m, HasConfig env) => SnapName -> Text -> m Text +buildBuildPlanUrl name file = do + urls <- asks (configUrls . getConfig) + return $ + case name of + LTS _ _ -> urlsLtsBuildPlans urls <> "/" <> file + Nightly _ -> urlsNightlyBuildPlans urls <> "/" <> file + gpdPackages :: [GenericPackageDescription] -> Map PackageName Version gpdPackages gpds = Map.fromList $ map (fromCabalIdent . C.package . C.packageDescription) gpds @@ -532,45 +536,35 @@ -> GenericPackageDescription -> (Map PackageName (Map FlagName Bool), DepErrors) selectPackageBuildPlan platform compiler pool gpd = - fromJust (go flagOptions Nothing) - where - go :: [Map FlagName Bool] -> Maybe (Map PackageName (Map FlagName Bool), DepErrors) -> Maybe (Map PackageName (Map FlagName Bool), DepErrors) - -- impossible - go [] Nothing = assert False Nothing - -- last - go [] (Just plan) = Just plan - -- got the best possible result - go _ (Just plan) | Map.null (snd plan) = Just plan - -- initial - go (flags:rest) Nothing = go rest $ Just (nextPlan flags) - -- keep looking for better results - go (flags:rest) (Just plan) = - go rest $ Just (betterPlan plan (nextPlan flags)) - - nextPlan flags = checkPackageBuildPlan platform compiler pool flags gpd - - betterPlan (f1, e1) (f2, e2) - | (Map.size e1) <= (Map.size e2) = (f1, e1) - | otherwise = (f2, e2) + (selectPlan . limitSearchSpace . NonEmpty.map makePlan) flagCombinations + where + selectPlan :: NonEmpty (a, DepErrors) -> (a, DepErrors) + selectPlan = F.foldr1 fewerErrors + where + fewerErrors p1 p2 + | nErrors p1 == 0 = p1 + | nErrors p1 <= nErrors p2 = p1 + | otherwise = p2 + where nErrors = Map.size . snd - flagName' = fromCabalFlagName . flagName + -- Avoid exponential complexity in flag combinations making us sad pandas. + -- See: https://github.com/commercialhaskell/stack/issues/543 + limitSearchSpace :: NonEmpty a -> NonEmpty a + limitSearchSpace (x :| xs) = x :| take (maxFlagCombinations - 1) xs + where maxFlagCombinations = 128 - -- Avoid exponential complexity in flag combinations making us sad pandas. - -- See: https://github.com/commercialhaskell/stack/issues/543 - maxFlagOptions = 128 + makePlan :: [(FlagName, Bool)] -> (Map PackageName (Map FlagName Bool), DepErrors) + makePlan flags = checkPackageBuildPlan platform compiler pool (Map.fromList flags) gpd - flagOptions :: [Map FlagName Bool] - flagOptions = take maxFlagOptions $ map Map.fromList $ mapM getOptions $ genPackageFlags gpd + flagCombinations :: NonEmpty [(FlagName, Bool)] + flagCombinations = mapM getOptions (genPackageFlags gpd) + where + getOptions :: C.Flag -> NonEmpty (FlagName, Bool) getOptions f - | flagManual f = [(flagName' f, flagDefault f)] - | flagDefault f = - [ (flagName' f, True) - , (flagName' f, False) - ] - | otherwise = - [ (flagName' f, False) - , (flagName' f, True) - ] + | flagManual f = (fname, flagDefault f) :| [] + | flagDefault f = (fname, True) :| [(fname, False)] + | otherwise = (fname, False) :| [(fname, True)] + where fname = (fromCabalFlagName . flagName) f -- | Check whether with the given set of flags a package's dependency -- constraints can be satisfied against a given build plan or pool of packages. @@ -711,32 +705,30 @@ ghcErrors = Map.filterWithKey isGhcWiredIn -- | Find a snapshot and set of flags that is compatible with and matches as --- best as possible with the given 'GenericPackageDescription's. Returns --- 'Nothing' if no such snapshot is found. +-- best as possible with the given 'GenericPackageDescription's. selectBestSnapshot :: ( MonadIO m, MonadCatch m, MonadLogger m, MonadReader env m , HasHttpManager env, HasConfig env, HasGHCVariant env , MonadBaseControl IO m) => [GenericPackageDescription] - -> [SnapName] + -> NonEmpty SnapName -> m (SnapName, BuildPlanCheck) selectBestSnapshot gpds snaps = do $logInfo $ "Selecting the best among " - <> T.pack (show (length snaps)) + <> T.pack (show (NonEmpty.length snaps)) <> " snapshots...\n" - loop Nothing snaps + F.foldr1 go (NonEmpty.map getResult snaps) where - loop Nothing [] = error "Bug: in best snapshot selection" - loop (Just pair) [] = return pair - loop bestYet (snap:rest) = do + go mold mnew = do + old@(_snap, bpc) <- mold + case bpc of + BuildPlanCheckOk {} -> return old + _ -> fmap (betterSnap old) mnew + + getResult snap = do result <- checkSnapBuildPlan gpds Nothing snap reportResult result snap - let new = (snap, result) - case result of - BuildPlanCheckOk {} -> return new - _ -> case bestYet of - Nothing -> loop (Just new) rest - Just old -> loop (Just (betterSnap old new)) rest + return (snap, result) betterSnap (s1, r1) (s2, r2) | compareBuildPlanCheck r1 r2 /= LT = (s1, r1) @@ -765,6 +757,21 @@ , "\n" ] +showPackageFlags :: PackageName -> Map FlagName Bool -> Text +showPackageFlags pkg fl = + if (not $ Map.null fl) then + T.concat + [ " - " + , T.pack $ packageNameString pkg + , ": " + , T.pack $ intercalate ", " + $ map formatFlags (Map.toList fl) + , "\n" + ] + else "" + where + formatFlags (f, v) = (show f) ++ " = " ++ (show v) + showMapPackages :: Map PackageName a -> Text showMapPackages mp = showItems $ Map.keys mp @@ -816,21 +823,7 @@ flagVals = T.concat (map showFlags userPkgs) userPkgs = Map.keys $ Map.unions (Map.elems (fmap deNeededBy errs)) - showFlags pkg = maybe "" (printFlags pkg) (Map.lookup pkg flags) - - printFlags pkg fl = - if (not $ Map.null fl) then - T.concat - [ " - " - , T.pack $ packageNameString pkg - , ": " - , T.pack $ intercalate ", " - $ map formatFlags (Map.toList fl) - , "\n" - ] - else "" - - formatFlags (f, v) = (show f) ++ " = " ++ (show v) + showFlags pkg = maybe "" (showPackageFlags pkg) (Map.lookup pkg flags) shadowMiniBuildPlan :: MiniBuildPlan -> Set PackageName diff --git a/src/Stack/Clean.hs b/src/Stack/Clean.hs --- a/src/Stack/Clean.hs +++ b/src/Stack/Clean.hs @@ -8,8 +8,7 @@ ) where import Control.Exception (Exception) -import Control.Monad (when) -import Control.Monad.Catch (MonadCatch, throwM) +import Control.Monad.Catch (MonadCatch, MonadThrow, throwM) import Control.Monad.IO.Class (MonadIO) import Control.Monad.Logger (MonadLogger) import Control.Monad.Reader (MonadReader, asks) @@ -18,57 +17,53 @@ import qualified Data.Map.Strict as Map import Data.Maybe (mapMaybe) import Data.Typeable (Typeable) +import Path (Path, Abs, Dir) import Path.IO (ignoringAbsence, removeDirRecur) import Stack.Build.Source (getLocalPackageViews) import Stack.Build.Target (LocalPackageView(..)) import Stack.Constants (distDirFromDir, workDirFromDir) -import Stack.Types (HasEnvConfig,PackageName, bcWorkDir, getBuildConfig) +import Stack.Types --- | Reset the build, i.e. remove the @dist@ directory --- (for example @.stack-work\/dist\/x84_64-linux\/Cabal-1.22.4.0@) --- for all targets. +-- | Deletes build artifacts in the current project. -- -- Throws 'StackCleanException'. clean :: (MonadCatch m, MonadIO m, MonadReader env m, HasEnvConfig env, MonadLogger m) => CleanOpts -> m () -clean (CleanTargets targets) = - cleanup targets False -clean (CleanFull _ ) = - cleanup [] True +clean cleanOpts = do + dirs <- dirsToDelete cleanOpts + forM_ dirs (ignoringAbsence . removeDirRecur) -cleanup - :: (MonadCatch m, MonadIO m, MonadReader env m, HasEnvConfig env, MonadLogger m) - => [PackageName] -> Bool - -> m() -cleanup targets doFullClean = do - locals <- getLocalPackageViews - case targets \\ Map.keys locals of - [] -> do - let lpvs = - if null targets - then Map.elems locals -- default to cleaning all local packages - else mapMaybe (`Map.lookup` locals) targets - forM_ lpvs $ \(LocalPackageView{lpvRoot = pkgDir},_) -> do - let delDir = - if doFullClean - then workDirFromDir pkgDir - else distDirFromDir pkgDir - ignoringAbsence . removeDirRecur =<< delDir - when doFullClean $ do - bconfig <- asks getBuildConfig - bcwd <- bcWorkDir bconfig - ignoringAbsence (removeDirRecur bcwd) - pkgs -> throwM (NonLocalPackages pkgs) +dirsToDelete + :: (MonadThrow m, MonadIO m, MonadReader env m, HasEnvConfig env, MonadLogger m) + => CleanOpts + -> m [Path Abs Dir] +dirsToDelete cleanOpts = do + localPkgDirs <- asks (Map.keys . envConfigPackages . getEnvConfig) + case cleanOpts of + CleanShallow [] -> do + mapM distDirFromDir localPkgDirs + CleanShallow targets -> do + localPkgViews <- getLocalPackageViews + let localPkgNames = Map.keys localPkgViews + getPkgDir pkgName = fmap (lpvRoot . fst) (Map.lookup pkgName localPkgViews) + case targets \\ localPkgNames of + [] -> mapM distDirFromDir (mapMaybe getPkgDir targets) + xs -> throwM (NonLocalPackages xs) + CleanFull -> do + pkgWorkDirs <- mapM workDirFromDir localPkgDirs + projectWorkDir <- getProjectWorkDir + return (projectWorkDir : pkgWorkDirs) --- | Options for cleaning a project. -data CleanOpts = CleanTargets - { cleanOptsTargets :: [PackageName] - -- ^ Names of the packages to clean. - -- If the list is empty, every local package should be cleaned. - } - | CleanFull { cleanOptsFull :: Bool } +-- | Options for @stack clean@. +data CleanOpts + = CleanShallow [PackageName] + -- ^ Delete the "dist directories" as defined in 'Stack.Constants.distRelativeDir' + -- for the given local packages. If no packages are given, all project packages + -- should be cleaned. + | CleanFull + -- ^ Delete all work directories in the project. -- | Exceptions during cleanup. newtype StackCleanException diff --git a/src/Stack/Config.hs b/src/Stack/Config.hs --- a/src/Stack/Config.hs +++ b/src/Stack/Config.hs @@ -42,7 +42,7 @@ import Control.Arrow ((***)) import Control.Exception (assert) import Control.Monad (liftM, unless, when, filterM) -import Control.Monad.Catch (MonadThrow, MonadCatch, catchAll, throwM) +import Control.Monad.Catch (MonadThrow, MonadCatch, MonadMask, catchAll, throwM) import Control.Monad.Extra (firstJustM) import Control.Monad.IO.Class import Control.Monad.Logger hiding (Loc) @@ -55,6 +55,7 @@ import qualified Data.ByteString.Lazy as L import Data.Foldable (forM_) import qualified Data.IntMap as IntMap +import Data.IORef (newIORef) import qualified Data.Map as Map import Data.Maybe import Data.Monoid @@ -76,11 +77,12 @@ import qualified Paths_stack as Meta import Safe (headMay) import Stack.BuildPlan +import Stack.Config.Build +import Stack.Config.Urls import Stack.Config.Docker import Stack.Config.Nix import Stack.Constants import qualified Stack.Image as Image -import Stack.PackageIndex import Stack.Types import Stack.Types.Internal import System.Environment @@ -135,12 +137,18 @@ stackRoot = configStackRoot config -- | Download the 'Snapshots' value from stackage.org. -getSnapshots :: (MonadThrow m, MonadIO m, MonadReader env m, HasHttpManager env, HasStackRoot env, HasConfig env) +getSnapshots :: (MonadThrow m, MonadMask m, MonadIO m, MonadReader env m, HasHttpManager env, HasStackRoot env, HasConfig env, MonadLogger m) => m Snapshots -getSnapshots = askLatestSnapshotUrl >>= parseUrl . T.unpack >>= downloadJSON +getSnapshots = do + latestUrlText <- askLatestSnapshotUrl + latestUrl <- parseUrl (T.unpack latestUrlText) + $logDebug $ "Downloading snapshot versions file from " <> latestUrlText + result <- downloadJSON latestUrl + $logDebug $ "Done downloading and parsing snapshot versions file" + return result -- | Turn an 'AbstractResolver' into a 'Resolver'. -makeConcreteResolver :: (MonadIO m, MonadReader env m, HasConfig env, MonadThrow m, HasHttpManager env, MonadLogger m) +makeConcreteResolver :: (MonadIO m, MonadReader env m, HasConfig env, MonadThrow m, MonadMask m, HasHttpManager env, MonadLogger m) => AbstractResolver -> m Resolver makeConcreteResolver (ARResolver r) = return r @@ -172,7 +180,7 @@ -- | Get the latest snapshot resolver available. getLatestResolver - :: (MonadIO m, MonadThrow m, MonadReader env m, HasConfig env, HasHttpManager env, MonadLogger m) + :: (MonadIO m, MonadThrow m, MonadMask m, MonadReader env m, HasConfig env, HasHttpManager env, MonadLogger m) => m Resolver getLatestResolver = do snapshots <- getSnapshots @@ -193,11 +201,14 @@ -> m Config configFromConfigMonoid configStackRoot configUserConfigPath mresolver mproject configMonoid@ConfigMonoid{..} = do configWorkDir <- parseRelDir (fromMaybe ".stack-work" configMonoidWorkDir) + -- This code is to handle the deprecation of latest-snapshot-url + configUrls <- case (configMonoidLatestSnapshotUrl, urlsMonoidLatestSnapshot configMonoidUrls) of + (Just url, Nothing) -> do + $logWarn "The latest-snapshot-url field is deprecated in favor of 'urls' configuration" + return (urlsFromMonoid configMonoidUrls) { urlsLatestSnapshot = url } + _ -> return (urlsFromMonoid configMonoidUrls) let configConnectionCount = fromMaybe 8 configMonoidConnectionCount configHideTHLoading = fromMaybe True configMonoidHideTHLoading - configLatestSnapshotUrl = fromMaybe - "https://s3.amazonaws.com/haddock.stackage.org/snapshots.json" - configMonoidLatestSnapshotUrl configPackageIndices = fromMaybe [PackageIndex { indexName = IndexName "Hackage" @@ -240,9 +251,10 @@ configPlatformVariant <- liftIO $ maybe PlatformVariantNone PlatformVariant <$> lookupEnv platformVariantEnvVar + let configBuild = buildOptsFromMonoid configMonoidBuildOpts configDocker <- dockerOptsFromMonoid (fmap fst mproject) configStackRoot mresolver configMonoidDockerOpts - configNix <- nixOptsFromMonoid (fmap fst mproject) configMonoidNixOpts os + configNix <- nixOptsFromMonoid configMonoidNixOpts os rawEnv <- liftIO getEnvironment pathsEnv <- augmentPathMap (map toFilePath configMonoidExtraPath) @@ -301,6 +313,10 @@ Just True -> return True _ -> getInContainer + configPackageCaches <- liftIO $ newIORef Nothing + + let configMaybeProject = mproject + return Config {..} -- | Get the default 'GHCVariant'. On older Linux systems with libgmp4, returns 'GHCGMP4'. @@ -308,7 +324,11 @@ :: (MonadIO m, MonadBaseControl IO m, MonadCatch m, MonadLogger m) => EnvOverride -> Platform -> m GHCVariant getDefaultGHCVariant menv (Platform _ Linux) = do + $logDebug "Checking whether stack was built with libgmp4" isGMP4 <- getIsGMP4 menv + if isGMP4 + then $logDebug "Stack was built with libgmp4, so the default ghc-variant will be gmp4" + else $logDebug "Stack was not built with libgmp4" return (if isGMP4 then GHCGMP4 else GHCStandard) getDefaultGHCVariant _ _ = return GHCStandard @@ -368,7 +388,7 @@ -- | Load the configuration, using current directory, environment variables, -- and defaults as necessary. -loadConfig :: (MonadLogger m,MonadIO m,MonadCatch m,MonadThrow m,MonadBaseControl IO m,MonadReader env m,HasHttpManager env,HasTerminal env) +loadConfig :: (MonadLogger m,MonadIO m,MonadMask m,MonadThrow m,MonadBaseControl IO m,MonadReader env m,HasHttpManager env,HasTerminal env) => ConfigMonoid -- ^ Config monoid from parsed command-line arguments -> Maybe (Path Abs File) @@ -377,9 +397,9 @@ -- ^ Override resolver -> m (LoadConfig m) loadConfig configArgs mstackYaml mresolver = do - (stackRoot, userOwnsStackRoot) <- determineStackRootAndOwnership + (stackRoot, userOwnsStackRoot) <- determineStackRootAndOwnership configArgs userConfigPath <- getDefaultUserConfigPath stackRoot - extraConfigs0 <- getExtraConfigs userConfigPath >>= mapM loadYaml + extraConfigs0 <- getExtraConfigs userConfigPath >>= mapM loadConfigYaml let extraConfigs = -- non-project config files' existence of a docker section should never default docker -- to enabled, so make it look like they didn't exist @@ -388,10 +408,6 @@ extraConfigs0 mproject <- loadProjectConfig mstackYaml - let printUserMessage (p, _, _) = - maybe (return ()) ($logWarn . T.pack) (projectUserMsg p) - maybe (return ()) printUserMessage mproject - let mproject' = (\(project, stackYaml, _) -> (project, stackYaml)) <$> mproject config <- configFromConfigMonoid stackRoot userConfigPath mresolver mproject' $ mconcat $ case mproject of @@ -415,7 +431,7 @@ -- | Load the build configuration, adds build-specific values to config loaded by @loadConfig@. -- values. -loadBuildConfig :: (MonadLogger m, MonadIO m, MonadCatch m, MonadReader env m, HasHttpManager env, MonadBaseControl IO m, HasTerminal env) +loadBuildConfig :: (MonadLogger m, MonadIO m, MonadMask m, MonadReader env m, HasHttpManager env, MonadBaseControl IO m, HasTerminal env) => Maybe (Project, Path Abs File, ConfigMonoid) -> Config -> Maybe AbstractResolver -- override resolver @@ -426,7 +442,9 @@ miniConfig <- loadMiniConfig config (project', stackYamlFP) <- case mproject of - Just (project, fp, _) -> return (project, fp) + Just (project, fp, _) -> do + forM_ (projectUserMsg project) ($logWarn . T.pack) + return (project, fp) Nothing -> do $logInfo "Run from outside a project, using implicit global project config" destDir <- getImplicitGlobalProjectDir config @@ -438,7 +456,7 @@ exists <- doesFileExist dest if exists then do - ProjectAndConfigMonoid project _ <- loadYaml dest + ProjectAndConfigMonoid project _ <- loadConfigYaml dest when (getTerminal env) $ case mresolver of Nothing -> @@ -508,8 +526,6 @@ extraPackageDBs <- mapM resolveDir' (projectExtraPackageDBs project) - packageCaches <- runReaderT (getMinimalEnvOverride >>= getPackageCaches) miniConfig - return BuildConfig { bcConfig = config , bcResolver = projectResolver project @@ -521,7 +537,6 @@ , bcFlags = projectFlags project , bcImplicitGlobal = isNothing mproject , bcGHCVariant = getGHCVariant miniConfig - , bcPackageCaches = packageCaches } -- | Resolve a PackageEntry into a list of paths, downloading and cloning as @@ -532,17 +547,14 @@ => EnvOverride -> Path Abs Dir -- ^ project root -> PackageEntry - -> m [(Path Abs Dir, Bool)] + -> m [(Path Abs Dir, TreatLikeExtraDep)] resolvePackageEntry menv projRoot pe = do entryRoot <- resolvePackageLocation menv projRoot (peLocation pe) paths <- case peSubdirs pe of [] -> return [entryRoot] subs -> mapM (resolveDir entryRoot) subs - case peValidWanted pe of - Nothing -> return () - Just _ -> $logWarn "Warning: you are using the deprecated valid-wanted field. You should instead use extra-dep. See: http://docs.haskellstack.org/en/stable/yaml_configuration/#packages" - return $ map (, not $ peExtraDep pe) paths + return $ map (, peExtraDep pe) paths -- | Resolve a PackageLocation into a path, downloading and cloning as -- necessary. @@ -577,16 +589,17 @@ unless exists $ do ignoringAbsence (removeDirRecur dirTmp) - let cloneAndExtract commandName resetCommand commit = do + let cloneAndExtract commandName cloneArgs resetCommand commit = do ensureDir (parent dirTmp) readInNull (parent dirTmp) commandName menv - [ "clone" - , T.unpack url + ("clone" : + cloneArgs ++ + [ T.unpack url , toFilePathNoTrailingSep dirTmp - ] + ]) Nothing readInNull dirTmp commandName menv - (resetCommand ++ [T.unpack commit]) + (resetCommand ++ [T.unpack commit, "--"]) (Just $ "Please ensure that commit " <> commit <> " exists within " <> url) @@ -601,7 +614,7 @@ liftIO $ withBinaryFile fp ReadMode $ \h -> do lbs <- L.hGetContents h let entries = Tar.read $ GZip.decompress lbs - Tar.unpack fp entries + Tar.unpack (toFilePath dirTmp) entries tryZip = do $logDebug $ "Trying to unzip " <> T.pack fp archive <- fmap Zip.toArchive $ liftIO $ L.readFile fp @@ -616,8 +629,8 @@ tryTar `catchAllLog` tryZip `catchAllLog` err - RPTGit commit -> cloneAndExtract "git" ["reset", "--hard"] commit - RPTHg commit -> cloneAndExtract "hg" ["update", "-C"] commit + RPTGit commit -> cloneAndExtract "git" ["--recursive"] ["reset", "--hard"] commit + RPTHg commit -> cloneAndExtract "hg" [] ["update", "-C"] commit renameDir dirTmp dir @@ -636,19 +649,24 @@ -- On Windows, the second value is always 'True'. determineStackRootAndOwnership :: (MonadIO m, MonadCatch m) - => m (Path Abs Dir, Bool) -determineStackRootAndOwnership = do + => ConfigMonoid + -- ^ Parsed command-line arguments + -> m (Path Abs Dir, Bool) +determineStackRootAndOwnership clArgs = do stackRoot <- do - mstackRoot <- liftIO $ lookupEnv stackRootEnvVar - case mstackRoot of - Nothing -> getAppUserDataDir stackProgName - Just x -> parseAbsDir x + case configMonoidStackRoot clArgs of + Just x -> return x + Nothing -> do + mstackRoot <- liftIO $ lookupEnv stackRootEnvVar + case mstackRoot of + Nothing -> getAppUserDataDir stackProgName + Just x -> parseAbsDir x (existingStackRootOrParentDir, userOwnsIt) <- do mdirAndOwnership <- findInParents getDirAndOwnership stackRoot case mdirAndOwnership of Just x -> return x - Nothing -> throwM (BadStackRootEnvVar stackRoot) + Nothing -> throwM (BadStackRoot stackRoot) when (existingStackRootOrParentDir /= stackRoot) $ if userOwnsIt @@ -729,15 +747,28 @@ $ fromMaybe userConfigPath mstackConfig : maybe [] return (mstackGlobalConfig <|> defaultStackGlobalConfigPath) +-- | Load and parse YAML from the given conig file. Throws +-- 'ParseConfigFileException' when there's a decoding error. +loadConfigYaml + :: (FromJSON (WithJSONWarnings a), MonadIO m, MonadLogger m) + => Path Abs File -> m a +loadConfigYaml path = do + eres <- loadYaml path + case eres of + Left err -> liftIO $ throwM (ParseConfigFileException path err) + Right res -> return res + -- | Load and parse YAML from the given file. -loadYaml :: (FromJSON (WithJSONWarnings a), MonadIO m, MonadLogger m) => Path Abs File -> m a +loadYaml + :: (FromJSON (WithJSONWarnings a), MonadIO m, MonadLogger m) + => Path Abs File -> m (Either Yaml.ParseException a) loadYaml path = do - WithJSONWarnings result warnings <- - liftIO $ - Yaml.decodeFileEither (toFilePath path) >>= - either (throwM . ParseConfigFileException path) return - logJSONWarnings (toFilePath path) warnings - return result + eres <- liftIO $ Yaml.decodeFileEither (toFilePath path) + case eres of + Left err -> return (Left err) + Right (WithJSONWarnings res warnings) -> do + logJSONWarnings (toFilePath path) warnings + return (Right res) -- | Get the location of the project config file, if it exists. getProjectConfig :: (MonadIO m, MonadThrow m, MonadLogger m) @@ -784,7 +815,7 @@ return Nothing where load fp = do - ProjectAndConfigMonoid project config <- loadYaml fp + ProjectAndConfigMonoid project config <- loadConfigYaml fp return $ Just (project, fp, config) -- | Get the location of the default stack configuration file. diff --git a/src/Stack/Config/Build.hs b/src/Stack/Config/Build.hs new file mode 100644 --- /dev/null +++ b/src/Stack/Config/Build.hs @@ -0,0 +1,68 @@ +{-# LANGUAGE RecordWildCards #-} + +-- | Build configuration +module Stack.Config.Build where + +import Data.Maybe (fromMaybe) +import Stack.Types + +-- | Interprets BuildOptsMonoid options. +buildOptsFromMonoid :: BuildOptsMonoid -> BuildOpts +buildOptsFromMonoid BuildOptsMonoid{..} = BuildOpts + { boptsLibProfile = fromMaybe + (boptsLibProfile defaultBuildOpts) + buildMonoidLibProfile + , boptsExeProfile = fromMaybe + (boptsExeProfile defaultBuildOpts) + buildMonoidExeProfile + , boptsHaddock = fromMaybe + (boptsHaddock defaultBuildOpts) + buildMonoidHaddock + , boptsOpenHaddocks = fromMaybe + (boptsOpenHaddocks defaultBuildOpts) + buildMonoidOpenHaddocks + , boptsHaddockDeps = buildMonoidHaddockDeps + , boptsInstallExes = fromMaybe + (boptsInstallExes defaultBuildOpts) + buildMonoidInstallExes + , boptsPreFetch = fromMaybe + (boptsPreFetch defaultBuildOpts) + buildMonoidPreFetch + , boptsKeepGoing = buildMonoidKeepGoing + , boptsForceDirty = fromMaybe + (boptsForceDirty defaultBuildOpts) + buildMonoidForceDirty + , boptsTests = fromMaybe (boptsTests defaultBuildOpts) buildMonoidTests + , boptsTestOpts = testOptsFromMonoid buildMonoidTestOpts + , boptsBenchmarks = fromMaybe + (boptsBenchmarks defaultBuildOpts) + buildMonoidBenchmarks + , boptsBenchmarkOpts = benchmarkOptsFromMonoid buildMonoidBenchmarkOpts + , boptsReconfigure = fromMaybe + (boptsReconfigure defaultBuildOpts) + buildMonoidReconfigure + , boptsCabalVerbose = fromMaybe + (boptsCabalVerbose defaultBuildOpts) + buildMonoidCabalVerbose + , boptsSplitObjs = fromMaybe + (boptsSplitObjs defaultBuildOpts) + buildMonoidSplitObjs + } + +testOptsFromMonoid :: TestOptsMonoid -> TestOpts +testOptsFromMonoid TestOptsMonoid{..} = + defaultTestOpts + { toRerunTests = fromMaybe (toRerunTests defaultTestOpts) toMonoidRerunTests + , toAdditionalArgs = toMonoidAdditionalArgs + , toCoverage = fromMaybe (toCoverage defaultTestOpts) toMonoidCoverage + , toDisableRun = fromMaybe (toDisableRun defaultTestOpts) toMonoidDisableRun + } + +benchmarkOptsFromMonoid :: BenchmarkOptsMonoid -> BenchmarkOpts +benchmarkOptsFromMonoid BenchmarkOptsMonoid{..} = + defaultBenchmarkOpts + { beoAdditionalArgs = beoMonoidAdditionalArgs + , beoDisableRun = fromMaybe + (beoDisableRun defaultBenchmarkOpts) + beoMonoidDisableRun + } diff --git a/src/Stack/Config/Nix.hs b/src/Stack/Config/Nix.hs --- a/src/Stack/Config/Nix.hs +++ b/src/Stack/Config/Nix.hs @@ -3,6 +3,7 @@ -- | Nix configuration module Stack.Config.Nix (nixOptsFromMonoid + ,nixCompiler ,StackNixException(..) ) where @@ -20,11 +21,10 @@ -- | Interprets NixOptsMonoid options. nixOptsFromMonoid :: (Monad m, MonadCatch m) - => Maybe Project - -> NixOptsMonoid + => NixOptsMonoid -> OS -> m NixOpts -nixOptsFromMonoid mproject NixOptsMonoid{..} os = do +nixOptsFromMonoid NixOptsMonoid{..} os = do let nixEnable = fromMaybe nixMonoidDefaultEnable nixMonoidEnable defaultPure = case os of OSX -> False @@ -34,21 +34,24 @@ nixInitFile = nixMonoidInitFile nixShellOptions = fromMaybe [] nixMonoidShellOptions ++ prefixAll (T.pack "-I") (fromMaybe [] nixMonoidPath) - nixCompiler resolverOverride compilerOverride = - let mresolver = resolverOverride <|> fmap projectResolver mproject - mcompiler = compilerOverride <|> join (fmap projectCompiler mproject) - in case (mresolver, mcompiler) of - (_, Just (GhcVersion v)) -> nixCompilerFromVersion v - (Just (ResolverCompiler (GhcVersion v)), _) -> nixCompilerFromVersion v - (Just (ResolverSnapshot (LTS x y)), _) -> - T.pack ("haskell.packages.lts-" ++ show x ++ "_" ++ show y ++ ".ghc") - _ -> T.pack "ghc" when (not (null nixPackages) && isJust nixInitFile) $ throwM NixCannotUseShellFileAndPackagesException return NixOpts{..} where prefixAll p (x:xs) = p : x : prefixAll p xs prefixAll _ _ = [] - nixCompilerFromVersion v = T.filter (/= '.') $ T.append (T.pack "haskell.compiler.ghc") (versionText v) + +nixCompiler :: Config -> Maybe Resolver -> Maybe CompilerVersion -> T.Text +nixCompiler config resolverOverride compilerOverride = + let mproject = fst <$> configMaybeProject config + mresolver = resolverOverride <|> fmap projectResolver mproject + mcompiler = compilerOverride <|> join (fmap projectCompiler mproject) + nixCompilerFromVersion v = T.filter (/= '.') $ T.append (T.pack "haskell.compiler.ghc") (versionText v) + in case (mresolver, mcompiler) of + (_, Just (GhcVersion v)) -> nixCompilerFromVersion v + (Just (ResolverCompiler (GhcVersion v)), _) -> nixCompilerFromVersion v + (Just (ResolverSnapshot (LTS x y)), _) -> + T.pack ("haskell.packages.lts-" ++ show x ++ "_" ++ show y ++ ".ghc") + _ -> T.pack "ghc" -- Exceptions thown specifically by Stack.Nix data StackNixException diff --git a/src/Stack/Config/Urls.hs b/src/Stack/Config/Urls.hs new file mode 100644 --- /dev/null +++ b/src/Stack/Config/Urls.hs @@ -0,0 +1,20 @@ +{-# LANGUAGE OverloadedStrings #-} + +module Stack.Config.Urls (urlsFromMonoid) where + +import Stack.Types +import Data.Maybe + +urlsFromMonoid :: UrlsMonoid -> Urls +urlsFromMonoid monoid = + Urls + (fromMaybe defaultLatestSnapshot $ urlsMonoidLatestSnapshot monoid) + (fromMaybe defaultLtsBuildPlans $ urlsMonoidLtsBuildPlans monoid) + (fromMaybe defaultNightlyBuildPlans $ urlsMonoidNightlyBuildPlans monoid) + where + defaultLatestSnapshot = + "https://www.stackage.org/download/snapshots.json" + defaultLtsBuildPlans = + "https://raw.githubusercontent.com/fpco/lts-haskell/master/" + defaultNightlyBuildPlans = + "https://raw.githubusercontent.com/fpco/stackage-nightly/master/" diff --git a/src/Stack/Constants.hs b/src/Stack/Constants.hs --- a/src/Stack/Constants.hs +++ b/src/Stack/Constants.hs @@ -4,28 +4,23 @@ -- | Constants used throughout the project. module Stack.Constants - (builtConfigFileFromDir - ,builtFileFromDir - ,buildPlanDir - ,configuredFileFromDir - ,defaultShakeThreads + (buildPlanDir ,distDirFromDir ,workDirFromDir ,distRelativeDir ,haskellModuleExts ,imageStagingDir ,projectDockerSandboxDir - ,rawGithubUrl ,stackDotYaml ,stackRootEnvVar + ,stackRootOptionName + ,deprecatedStackRootOptionName ,inContainerEnvVar - ,userDocsDir ,configCacheFile ,configCabalMod ,buildCacheFile ,testSuccessFile ,testBuiltFile - ,benchBuiltFile ,stackProgName ,stackProgNameUpper ,wiredInPackages @@ -51,7 +46,6 @@ import Data.HashSet (HashSet) import qualified Data.HashSet as HashSet import Data.Text (Text) -import qualified Data.Text as T import Path as FL import Prelude import Stack.Types.Compiler @@ -71,58 +65,6 @@ haskellPreprocessorExts :: [Text] haskellPreprocessorExts = ["gc", "chs", "hsc", "x", "y", "ly", "cpphs"] --- | The filename used for completed build indicators. -builtFileFromDir :: (MonadThrow m, MonadReader env m, HasPlatform env,HasEnvConfig env) - => Path Abs Dir - -> m (Path Abs File) -builtFileFromDir fp = do - dist <- distDirFromDir fp - return (dist $(mkRelFile "stack.gen")) - --- | The filename used for completed configure indicators. -configuredFileFromDir :: (MonadThrow m, MonadReader env m, HasPlatform env,HasEnvConfig env) - => Path Abs Dir - -> m (Path Abs File) -configuredFileFromDir fp = do - dist <- distDirFromDir fp - return (dist $(mkRelFile "setup-config")) - --- | The filename used for completed build indicators. -builtConfigFileFromDir :: (MonadThrow m, MonadReader env m, HasPlatform env,HasEnvConfig env) - => Path Abs Dir - -> m (Path Abs File) -builtConfigFileFromDir fp = - liftM (fp ) builtConfigRelativeFile - --- | Relative location of completed build indicators. -builtConfigRelativeFile :: (MonadThrow m, MonadReader env m, HasPlatform env,HasEnvConfig env) - => m (Path Rel File) -builtConfigRelativeFile = do - dist <- distRelativeDir - return (dist $(mkRelFile "stack.config")) - --- | Default shake thread count for parallel builds. -defaultShakeThreads :: Int -defaultShakeThreads = 4 - --- -- | Hoogle database file. --- hoogleDatabaseFile :: Path Abs Dir -> Path Abs File --- hoogleDatabaseFile docLoc = --- docLoc --- $(mkRelFile "default.hoo") - --- -- | Extension for hoogle databases. --- hoogleDbExtension :: String --- hoogleDbExtension = "hoo" - --- -- | Extension of haddock files --- haddockExtension :: String --- haddockExtension = "haddock" - --- | User documentation directory. -userDocsDir :: Config -> Path Abs Dir -userDocsDir config = configStackRoot config $(mkRelDir "doc/") - -- | Output .o/.hi directory. objectInterfaceDir :: (MonadReader env m, HasConfig env) => BuildConfig -> m (Path Abs Dir) @@ -157,15 +99,6 @@ ( $(mkRelFile "stack-test-built")) (distDirFromDir dir) --- | The filename used to mark benchmarks as having built -benchBuiltFile :: (MonadThrow m, MonadReader env m, HasPlatform env,HasEnvConfig env) - => Path Abs Dir -- ^ Package directory - -> m (Path Abs File) -benchBuiltFile dir = - liftM - ( $(mkRelFile "stack-bench-built")) - (distDirFromDir dir) - -- | The filename used for dirtiness check of config. configCacheFile :: (MonadThrow m, MonadReader env m, HasPlatform env,HasEnvConfig env) => Path Abs Dir -- ^ Package directory. @@ -236,37 +169,6 @@ $(mkRelDir "dist") platformAndCabal --- | Get a URL for a raw file on Github -rawGithubUrl :: Text -- ^ user/org name - -> Text -- ^ repo name - -> Text -- ^ branch name - -> Text -- ^ filename - -> Text -rawGithubUrl org repo branch file = T.concat - [ "https://raw.githubusercontent.com/" - , org - , "/" - , repo - , "/" - , branch - , "/" - , file - ] - --- -- | Hoogle database file. --- hoogleDatabaseFile :: Path Abs Dir -> Path Abs File --- hoogleDatabaseFile docLoc = --- docLoc --- $(mkRelFile "default.hoo") - --- -- | Extension for hoogle databases. --- hoogleDbExtension :: String --- hoogleDbExtension = "hoo" - --- -- | Extension of haddock files --- haddockExtension :: String --- haddockExtension = "haddock" - -- | Docker sandbox from project root. projectDockerSandboxDir :: (MonadReader env m, HasConfig env) => Path Abs Dir -- ^ Project root @@ -300,6 +202,19 @@ -- | Environment variable used to override the '~/.stack' location. stackRootEnvVar :: String stackRootEnvVar = "STACK_ROOT" + +-- | Option name for the global stack root. +stackRootOptionName :: String +stackRootOptionName = "stack-root" + +-- | Deprecated option name for the global stack root. +-- +-- Deprecated since stack-1.1.0. +-- +-- TODO: Remove occurences of this variable and use 'stackRootOptionName' only +-- after an appropriate deprecation period. +deprecatedStackRootOptionName :: String +deprecatedStackRootOptionName = "global-stack-root" -- | Environment variable used to indicate stack is running in container. inContainerEnvVar :: String diff --git a/src/Stack/Constants.hs-boot b/src/Stack/Constants.hs-boot deleted file mode 100644 --- a/src/Stack/Constants.hs-boot +++ /dev/null @@ -1,3 +0,0 @@ -module Stack.Constants where - -stackRootEnvVar :: String diff --git a/src/Stack/Coverage.hs b/src/Stack/Coverage.hs --- a/src/Stack/Coverage.hs +++ b/src/Stack/Coverage.hs @@ -218,9 +218,8 @@ $logWarn $ "Since --all is used, it is redundant to specify these targets: " <> T.pack (show targetNames) (_,_,targets) <- parseTargetsFromBuildOpts AllowNoTargets - defaultBuildOpts - { boptsTargets = if hroptsAll opts then [] else targetNames - } + defaultBuildOptsCLI + { boptsCLITargets = if hroptsAll opts then [] else targetNames } liftM concat $ forM (Map.toList targets) $ \(name, target) -> case target of STUnknown -> fail $ diff --git a/src/Stack/Dot.hs b/src/Stack/Dot.hs --- a/src/Stack/Dot.hs +++ b/src/Stack/Dot.hs @@ -89,7 +89,7 @@ => DotOpts -> m (Map PackageName (Set PackageName, Maybe Version)) createDependencyGraph dotOpts = do - (_,_,locals,_,sourceMap) <- loadSourceMap NeedTargets defaultBuildOpts + (_,_,locals,_,sourceMap) <- loadSourceMap NeedTargets defaultBuildOptsCLI let graph = Map.fromList (localDependencies dotOpts locals) menv <- getMinimalEnvOverride installedMap <- fmap snd . fst4 <$> getInstalled menv diff --git a/src/Stack/Exec.hs b/src/Stack/Exec.hs --- a/src/Stack/Exec.hs +++ b/src/Stack/Exec.hs @@ -14,16 +14,16 @@ import Control.Monad.Trans.Control (MonadBaseControl) import Stack.Types import System.Process.Log -import System.Process.Read (EnvOverride) -#ifdef WINDOWS import Control.Exception.Lifted import Data.Streaming.Process (ProcessExitedUnsuccessfully(..)) import System.Exit import System.Process.Run (callProcess, Cmd(..)) +#ifdef WINDOWS +import System.Process.Read (EnvOverride) #else -import System.Process.Read (envHelper, preProcess) import System.Posix.Process (executeFile) +import System.Process.Read (EnvOverride, envHelper, preProcess) #endif -- | Default @EnvSettings@ which includes locals and GHC_PACKAGE_PATH @@ -45,16 +45,33 @@ } -- | Execute a process within the Stack configured environment. +-- +-- Execution will not return, because either: +-- +-- 1) On non-windows, execution is taken over by execv of the +-- sub-process. This allows signals to be propagated (#527) +-- +-- 2) On windows, an 'ExitCode' exception will be thrown. exec :: (MonadIO m, MonadLogger m, MonadThrow m, MonadBaseControl IO m) => EnvOverride -> String -> [String] -> m b +#ifdef WINDOWS +exec = execSpawn +#else exec menv cmd0 args = do $logProcessRun cmd0 args -#ifdef WINDOWS + cmd <- preProcess Nothing menv cmd0 + liftIO $ executeFile cmd True args (envHelper menv) +#endif + +-- | Like 'exec', but does not use 'execv' on non-windows. This way, there +-- is a sub-process, which is helpful in some cases (#1306) +-- +-- This function only exits by throwing 'ExitCode'. +execSpawn :: (MonadIO m, MonadLogger m, MonadThrow m, MonadBaseControl IO m) + => EnvOverride -> String -> [String] -> m b +execSpawn menv cmd0 args = do + $logProcessRun cmd0 args e <- try (callProcess (Cmd Nothing cmd0 menv args)) liftIO $ case e of Left (ProcessExitedUnsuccessfully _ ec) -> exitWith ec Right () -> exitSuccess -#else - cmd <- preProcess Nothing menv cmd0 - liftIO $ executeFile cmd True args (envHelper menv) -#endif diff --git a/src/Stack/Fetch.hs b/src/Stack/Fetch.hs --- a/src/Stack/Fetch.hs +++ b/src/Stack/Fetch.hs @@ -48,8 +48,6 @@ import Data.Either (partitionEithers) import qualified Data.Foldable as F import Data.Function (fix) -import Data.IORef (newIORef, readIORef, - writeIORef) import Data.List (intercalate) import Data.List.NonEmpty (NonEmpty) import qualified Data.List.NonEmpty as NE @@ -196,7 +194,7 @@ go >>= either throwM return Right x -> return x where - go = r <$> resolvePackagesAllowMissing menv idents0 names0 + go = r <$> resolvePackagesAllowMissing idents0 names0 r (missingNames, missingIdents, idents) | not $ Set.null missingNames = Left $ UnknownPackageNames missingNames | not $ Set.null missingIdents = Left $ UnknownPackageIdentifiers missingIdents "" @@ -204,12 +202,11 @@ resolvePackagesAllowMissing :: (MonadIO m, MonadReader env m, HasHttpManager env, HasConfig env, MonadLogger m, MonadThrow m, MonadBaseControl IO m, MonadCatch m) - => EnvOverride - -> Set PackageIdentifier + => Set PackageIdentifier -> Set PackageName -> m (Set PackageName, Set PackageIdentifier, Map PackageIdentifier ResolvedPackage) -resolvePackagesAllowMissing menv idents0 names0 = do - caches <- getPackageCaches menv +resolvePackagesAllowMissing idents0 names0 = do + caches <- getPackageCaches let versions = Map.fromListWith max $ map toTuple $ Map.keys caches (missingNames, idents1) = partitionEithers $ map (\name -> maybe (Left name ) (Right . PackageIdentifier name) @@ -263,35 +260,37 @@ -- | Provide a function which will load up a cabal @ByteString@ from the -- package indices. withCabalLoader - :: (MonadThrow m, MonadIO m, MonadReader env m, HasConfig env, MonadLogger m, HasHttpManager env, MonadBaseControl IO m, MonadCatch m) + :: (MonadIO m, MonadReader env m, HasConfig env, MonadLogger m, HasHttpManager env, MonadBaseControl IO m, MonadCatch m) => EnvOverride -> ((PackageIdentifier -> IO ByteString) -> m a) -> m a withCabalLoader menv inner = do - icaches <- getPackageCaches menv >>= liftIO . newIORef env <- ask -- Want to try updating the index once during a single run for missing -- package identifiers. We also want to ensure we only update once at a -- time + -- + -- TODO: probably makes sense to move this concern into getPackageCaches updateRef <- liftIO $ newMVar True + loadCaches <- getPackageCachesIO runInBase <- liftBaseWith $ \run -> return (void . run) -- TODO in the future, keep all of the necessary @Handle@s open let doLookup :: PackageIdentifier -> IO ByteString doLookup ident = do - cachesCurr <- liftIO $ readIORef icaches - eres <- lookupPackageIdentifierExact ident env cachesCurr + caches <- loadCaches + eres <- lookupPackageIdentifierExact ident env caches case eres of Just bs -> return bs -- Update the cache and try again Nothing -> do - let fuzzy = fuzzyLookupCandidates ident cachesCurr + let fuzzy = fuzzyLookupCandidates ident caches suggestions = case fuzzy of Nothing -> - case typoCorrectionCandidates ident cachesCurr of + case typoCorrectionCandidates ident caches of Nothing -> "" Just cs -> "Perhaps you meant " <> orSeparated cs <> "?" @@ -308,8 +307,8 @@ , "Updating and trying again." ] updateAllIndices menv - caches <- getPackageCaches menv - liftIO $ writeIORef icaches caches + _ <- getPackageCaches + return () return (False, doLookup ident) else return (toUpdate, throwM $ UnknownPackageIdentifiers diff --git a/src/Stack/GhcPkg.hs b/src/Stack/GhcPkg.hs --- a/src/Stack/GhcPkg.hs +++ b/src/Stack/GhcPkg.hs @@ -45,6 +45,7 @@ getGlobalDB :: (MonadIO m, MonadLogger m, MonadBaseControl IO m, MonadCatch m) => EnvOverride -> WhichCompiler -> m (Path Abs Dir) getGlobalDB menv wc = do + $logDebug "Getting global package database location" -- This seems like a strange way to get the global package database -- location, but I don't know of a better one bs <- ghcPkg menv wc [] ["list", "--global"] >>= either throwM return @@ -160,13 +161,14 @@ -- | Get the version of Cabal from the global package database. getCabalPkgVer :: (MonadThrow m, MonadIO m, MonadLogger m, MonadBaseControl IO m, MonadCatch m) => EnvOverride -> WhichCompiler -> m Version -getCabalPkgVer menv wc = - findGhcPkgVersion +getCabalPkgVer menv wc = do + $logDebug "Getting Cabal package version" + mres <- findGhcPkgVersion menv wc [] -- global DB - cabalPackageName >>= - maybe (throwM $ Couldn'tFindPkgId cabalPackageName) return + cabalPackageName + maybe (throwM $ Couldn'tFindPkgId cabalPackageName) return mres -- | Get the value for GHC_PACKAGE_PATH mkGhcPackagePath :: Bool -> Path Abs Dir -> Path Abs Dir -> [Path Abs Dir] -> Path Abs Dir -> Text diff --git a/src/Stack/Ghci.hs b/src/Stack/Ghci.hs --- a/src/Stack/Ghci.hs +++ b/src/Stack/Ghci.hs @@ -71,7 +71,7 @@ , ghciLoadLocalDeps :: !Bool , ghciSkipIntermediate :: !Bool , ghciHidePackages :: !Bool - , ghciBuildOpts :: !BuildOpts + , ghciBuildOptsCLI :: !BuildOptsCLI } deriving Show -- | Necessary information to load a package or its components. @@ -105,14 +105,11 @@ -- given options and configure it with the load paths and extensions -- of those targets. ghci - :: (HasConfig r, HasBuildConfig r, HasHttpManager r, MonadMask m, HasLogLevel r, HasTerminal r, HasEnvConfig r, MonadReader r m, MonadIO m, MonadThrow m, MonadLogger m, MonadCatch m, MonadBaseControl IO m) + :: (HasConfig r, HasBuildConfig r, HasHttpManager r, MonadMask m, HasLogLevel r, HasTerminal r, HasEnvConfig r, MonadReader r m, MonadIO m, MonadThrow m, MonadLogger m, MonadBaseControl IO m) => GhciOpts -> m () ghci opts@GhciOpts{..} = do - let bopts = ghciBuildOpts - { boptsTestOpts = (boptsTestOpts ghciBuildOpts) { toDisableRun = True } - , boptsBenchmarkOpts = (boptsBenchmarkOpts ghciBuildOpts) { beoDisableRun = True } - } - (targets,mainIsTargets,pkgs) <- ghciSetup opts { ghciBuildOpts = bopts } + bopts <- asks (configBuild . getConfig) + (targets,mainIsTargets,pkgs) <- ghciSetup opts config <- asks getConfig bconfig <- asks getBuildConfig wc <- getWhichCompiler @@ -136,10 +133,9 @@ T.unwords (map T.pack (nubOrd omittedOpts))) allModules <- checkForDuplicateModules ghciNoLoadModules pkgs oiDir <- objectInterfaceDir bconfig - (modulesToLoad, thingsToLoad) <- if ghciNoLoadModules then return ([], []) else do + (modulesToLoad, mainFile) <- if ghciNoLoadModules then return ([], Nothing) else do mainFile <- figureOutMainFile bopts mainIsTargets targets pkgs - let thingsToLoad = maybe [] (return . toFilePath) mainFile <> allModules - return (allModules, thingsToLoad) + return (allModules, mainFile) let odir = [ "-odir=" <> toFilePathNoTrailingSep oiDir , "-hidir=" <> toFilePathNoTrailingSep oiDir ] @@ -148,7 +144,7 @@ T.intercalate ", " (map (packageNameText . ghciPkgName) pkgs)) let execGhci extras = do menv <- liftIO $ configEnvOverride config defaultEnvSettings - exec menv + execSpawn menv (fromMaybe (compilerExeName wc) ghciGhcCommand) ("--interactive" : -- This initial "-i" resets the include directories to not @@ -163,9 +159,10 @@ else do let scriptPath = tmpDir $(mkRelFile "ghci-script") fp = toFilePath scriptPath - loadModules = ":load " <> unwords (map show thingsToLoad) + loadModules = ":load " <> unwords (map show modulesToLoad) + addMainFile = maybe "" ((":add " <>) . toFilePath) mainFile bringIntoScope = ":module + " <> unwords modulesToLoad - liftIO (writeFile fp (unlines [loadModules,bringIntoScope])) + liftIO (writeFile fp (unlines [loadModules,addMainFile,bringIntoScope])) setScriptPerms fp execGhci (macrosOpts ++ ["-ghci-script=" <> fp]) @@ -234,7 +231,7 @@ case elemIndex selected candidateIndices of Nothing -> do putStrLn - "Not loading any maim modules, as no valid module selected" + "Not loading any main modules, as no valid module selected" putStrLn "" return Nothing Just op -> do @@ -259,35 +256,34 @@ -- | Create a list of infos for each target containing necessary -- information to load that package/components. ghciSetup - :: (HasConfig r, HasHttpManager r, HasBuildConfig r, MonadMask m, HasTerminal r, HasLogLevel r, HasEnvConfig r, MonadReader r m, MonadIO m, MonadThrow m, MonadLogger m, MonadCatch m, MonadBaseControl IO m) + :: (HasConfig r, HasHttpManager r, HasBuildConfig r, MonadMask m, HasTerminal r, HasLogLevel r, HasEnvConfig r, MonadReader r m, MonadIO m, MonadThrow m, MonadLogger m, MonadBaseControl IO m) => GhciOpts -> m (Map PackageName SimpleTarget, Maybe (Map PackageName SimpleTarget), [GhciPkgInfo]) ghciSetup GhciOpts{..} = do - let bopts0 = ghciBuildOpts - (_,_,targets) <- parseTargetsFromBuildOpts AllowNoTargets bopts0 + (_,_,targets) <- parseTargetsFromBuildOpts AllowNoTargets ghciBuildOptsCLI mainIsTargets <- case ghciMainIs of Nothing -> return Nothing Just target -> do - (_,_,targets') <- parseTargetsFromBuildOpts AllowNoTargets bopts0 { boptsTargets = [target] } + (_,_,targets') <- parseTargetsFromBuildOpts AllowNoTargets ghciBuildOptsCLI { boptsCLITargets = [target] } return (Just targets') addPkgs <- forM ghciAdditionalPackages $ \name -> do let mres = (packageIdentifierName <$> parsePackageIdentifierFromString name) <|> parsePackageNameFromString name maybe (throwM $ InvalidPackageOption name) return mres - let bopts = bopts0 - { boptsTargets = boptsTargets bopts0 ++ map T.pack ghciAdditionalPackages + let boptsCli = ghciBuildOptsCLI + { boptsCLITargets = boptsCLITargets ghciBuildOptsCLI ++ map T.pack ghciAdditionalPackages } -- Try to build, but optimistically launch GHCi anyway if it fails (#1065) unless ghciNoBuild $ do - eres <- tryAny $ build (const (return ())) Nothing bopts + eres <- tryAny $ build (const (return ())) Nothing boptsCli case eres of Right () -> return () Left err -> do $logError $ T.pack (show err) $logWarn "Warning: build failed, but optimistically launching GHCi anyway" econfig <- asks getEnvConfig - (realTargets,_,_,_,sourceMap) <- loadSourceMap AllowNoTargets bopts + (realTargets,_,_,_,sourceMap) <- loadSourceMap AllowNoTargets boptsCli menv <- getMinimalEnvOverride (installedMap, _, _, _) <- getInstalled menv @@ -298,15 +294,15 @@ sourceMap directlyWanted <- forMaybeM (M.toList (envConfigPackages econfig)) $ - \(dir,validWanted) -> + \(dir,treatLikeExtraDep) -> do cabalfp <- findOrGenerateCabalFile dir name <- parsePackageNameFromFilePath cabalfp - if validWanted - then case M.lookup name targets of + if treatLikeExtraDep + then return Nothing + else case M.lookup name targets of Just simpleTargets -> return (Just (name, (cabalfp, simpleTargets))) Nothing -> return Nothing - else return Nothing let extraLoadDeps = getExtraLoadDeps ghciLoadLocalDeps sourceMap directlyWanted wanted <- if (ghciSkipIntermediate && not ghciLoadLocalDeps) || null extraLoadDeps @@ -331,7 +327,7 @@ infos <- forM wanted $ \(name,(cabalfp,target)) -> - makeGhciPkgInfo bopts sourceMap installedMap localLibs addPkgs name cabalfp target + makeGhciPkgInfo boptsCli sourceMap installedMap localLibs addPkgs name cabalfp target checkForIssues infos return (realTargets, mainIsTargets, infos) where @@ -344,7 +340,7 @@ -- | Make information necessary to load the given package in GHCi. makeGhciPkgInfo :: (MonadReader r m, HasEnvConfig r, MonadLogger m, MonadIO m, MonadCatch m) - => BuildOpts + => BuildOptsCLI -> SourceMap -> InstalledMap -> [PackageName] @@ -353,14 +349,15 @@ -> Path Abs File -> SimpleTarget -> m GhciPkgInfo -makeGhciPkgInfo bopts sourceMap installedMap locals addPkgs name cabalfp target = do +makeGhciPkgInfo boptsCli sourceMap installedMap locals addPkgs name cabalfp target = do + bopts <- asks (configBuild . getConfig) econfig <- asks getEnvConfig bconfig <- asks getBuildConfig let config = PackageConfig { packageConfigEnableTests = True , packageConfigEnableBenchmarks = True - , packageConfigFlags = localFlags (boptsFlags bopts) bconfig name + , packageConfigFlags = localFlags (boptsCLIFlags boptsCli) bconfig name , packageConfigCompilerVersion = envConfigCompilerVersion econfig , packageConfigPlatform = configPlatform (getConfig bconfig) } diff --git a/src/Stack/Ide.hs b/src/Stack/Ide.hs --- a/src/Stack/Ide.hs +++ b/src/Stack/Ide.hs @@ -39,16 +39,16 @@ -- given options and configure it with the load paths and extensions -- of those targets. ide - :: (HasConfig r, HasBuildConfig r, HasTerminal r, HasLogLevel r, MonadMask m, HasEnvConfig r, MonadReader r m, MonadIO m, MonadThrow m, MonadLogger m, MonadCatch m, MonadBaseControl IO m, HasHttpManager r) + :: (HasConfig r, HasBuildConfig r, HasTerminal r, HasLogLevel r, MonadMask m, HasEnvConfig r, MonadReader r m, MonadIO m, MonadThrow m, MonadLogger m, MonadBaseControl IO m, HasHttpManager r) => [Text] -- ^ Targets. -> [String] -- ^ GHC options. -> m () ide targets useropts = do - let bopts = defaultBuildOpts - { boptsTargets = targets - , boptsBuildSubset = BSOnlyDependencies + let boptsCli = defaultBuildOptsCLI + { boptsCLITargets = targets + , boptsCLIBuildSubset = BSOnlyDependencies } - (_realTargets,_,pkgs) <- ghciSetup (ideGhciOpts bopts) + (_realTargets,_,pkgs) <- ghciSetup (ideGhciOpts boptsCli) pwd <- getCurrentDir (pkgopts,_srcfiles) <- liftM mconcat $ forM pkgs $ getPackageOptsAndTargetFiles pwd @@ -110,8 +110,8 @@ (S.toList (ghciPkgCFiles pkg) <> S.toList (ghciPkgModFiles pkg) <> [paths_foo | paths_foo_exists])) -ideGhciOpts :: BuildOpts -> GhciOpts -ideGhciOpts bopts = GhciOpts +ideGhciOpts :: BuildOptsCLI -> GhciOpts +ideGhciOpts boptsCli = GhciOpts { ghciNoBuild = False , ghciArgs = [] , ghciGhcCommand = Nothing @@ -121,5 +121,5 @@ , ghciLoadLocalDeps = False , ghciSkipIntermediate = False , ghciHidePackages = True - , ghciBuildOpts = bopts + , ghciBuildOptsCLI = boptsCli } diff --git a/src/Stack/Image.hs b/src/Stack/Image.hs --- a/src/Stack/Image.hs +++ b/src/Stack/Image.hs @@ -1,11 +1,9 @@ -{-# LANGUAGE ConstraintKinds #-} +{-# LANGUAGE ConstraintKinds #-} {-# LANGUAGE DeriveDataTypeable #-} -{-# LANGUAGE FlexibleContexts #-} -{-# LANGUAGE KindSignatures #-} -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE RankNTypes #-} -{-# LANGUAGE RecordWildCards #-} -{-# LANGUAGE TemplateHaskell #-} +{-# LANGUAGE FlexibleContexts #-} +{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE RecordWildCards #-} +{-# LANGUAGE TemplateHaskell #-} -- | This module builds Docker (OpenContainer) images. module Stack.Image @@ -24,6 +22,8 @@ import qualified Data.Map.Strict as Map import Data.Maybe import Data.Typeable +import Data.Text (Text) +import qualified Data.Text as T import Path import Path.Extra import Path.IO @@ -34,73 +34,98 @@ type Build e m = (HasBuildConfig e, HasConfig e, HasEnvConfig e, HasTerminal e, MonadBaseControl IO m, MonadCatch m, MonadIO m, MonadLogger m, MonadReader e m) -type Assemble e m = (HasConfig e, HasTerminal e, MonadBaseControl IO m, MonadCatch m, MonadIO m, MonadLogger m, MonadMask m, MonadReader e m) +type Assemble e m = (HasConfig e, HasTerminal e, MonadBaseControl IO m, MonadIO m, MonadLogger m, MonadMask m, MonadReader e m) -- | Stages the executables & additional content in a staging -- directory under '.stack-work' -stageContainerImageArtifacts :: Build e m => m () -stageContainerImageArtifacts = do +stageContainerImageArtifacts + :: Build e m + => Maybe (Path Abs Dir) -> m () +stageContainerImageArtifacts mProjectRoot = do config <- asks getConfig - workingDir <- getCurrentDir - forM_ (zip [0..] $ imgDockers $ configImage config) $ \(idx, opts) -> do - imageDir <- imageStagingDir workingDir idx - ignoringAbsence (removeDirRecur imageDir) - ensureDir imageDir - stageExesInDir opts imageDir - syncAddContentToDir opts imageDir + forM_ + (zip [0 ..] (imgDockers $ configImage config)) + (\(idx,opts) -> + do imageDir <- + imageStagingDir (fromMaybeProjectRoot mProjectRoot) idx + ignoringAbsence (removeDirRecur imageDir) + ensureDir imageDir + stageExesInDir opts imageDir + syncAddContentToDir opts imageDir) -- | Builds a Docker (OpenContainer) image extending the `base` image -- specified in the project's stack.yaml. Then new image will be -- extended with an ENTRYPOINT specified for each `entrypoint` listed -- in the config file. -createContainerImageFromStage :: Assemble e m => m () -createContainerImageFromStage = do +createContainerImageFromStage + :: Assemble e m + => Maybe (Path Abs Dir) -> [Text] -> m () +createContainerImageFromStage mProjectRoot imageNames = do config <- asks getConfig - workingDir <- getCurrentDir - forM_ (zip [0..] $ imgDockers $ configImage config) $ \(idx, opts) -> do - imageDir <- imageStagingDir workingDir idx - createDockerImage opts imageDir - extendDockerImageWithEntrypoint opts imageDir + forM_ + (zip + [0 ..] + (filterImages + (map T.unpack imageNames) + (imgDockers $ configImage config))) + (\(idx,opts) -> + do imageDir <- + imageStagingDir (fromMaybeProjectRoot mProjectRoot) idx + createDockerImage opts imageDir + extendDockerImageWithEntrypoint opts imageDir) + where + filterImages [] = id -- all: no filter + filterImages names = filter (imageNameFound names . imgDockerImageName) + imageNameFound names (Just name) = name `elem` names + imageNameFound _ _ = False -- | Stage all the Package executables in the usr/local/bin -- subdirectory of a temp directory. -stageExesInDir :: Build e m => ImageDockerOpts -> Path Abs Dir -> m () +stageExesInDir + :: Build e m + => ImageDockerOpts -> Path Abs Dir -> m () stageExesInDir opts dir = do - srcBinPath <- - liftM ( $(mkRelDir "bin")) installationRootLocal - let destBinPath = dir - $(mkRelDir "usr/local/bin") + srcBinPath <- fmap ( $(mkRelDir "bin")) installationRootLocal + let destBinPath = dir $(mkRelDir "usr/local/bin") ensureDir destBinPath case imgDockerExecutables opts of Nothing -> copyDirRecur srcBinPath destBinPath - Just exes -> forM_ exes $ \exe -> do - exeRelFile <- parseRelFile exe - copyFile (srcBinPath exeRelFile) (destBinPath exeRelFile) + Just exes -> + forM_ + exes + (\exe -> + do exeRelFile <- parseRelFile exe + copyFile + (srcBinPath exeRelFile) + (destBinPath exeRelFile)) -- | Add any additional files into the temp directory, respecting the -- (Source, Destination) mapping. -syncAddContentToDir :: Build e m => ImageDockerOpts -> Path Abs Dir -> m () +syncAddContentToDir + :: Build e m + => ImageDockerOpts -> Path Abs Dir -> m () syncAddContentToDir opts dir = do bconfig <- asks getBuildConfig let imgAdd = imgDockerAdd opts forM_ (Map.toList imgAdd) (\(source,dest) -> - do sourcePath <- parseRelDir source + do sourcePath <- resolveDir (bcRoot bconfig) source destPath <- parseAbsDir dest let destFullPath = dir dropRoot destPath ensureDir destFullPath - copyDirRecur - (bcRoot bconfig sourcePath) - destFullPath) + copyDirRecur sourcePath destFullPath) -- | Derive an image name from the project directory. -imageName :: Path Abs Dir -> String +imageName + :: Path Abs Dir -> String imageName = map toLower . toFilePathNoTrailingSep . dirname -- | Create a general purpose docker image from the temporary -- directory of executables & static content. -createDockerImage :: Assemble e m => ImageDockerOpts -> Path Abs Dir -> m () +createDockerImage + :: Assemble e m + => ImageDockerOpts -> Path Abs Dir -> m () createDockerImage dockerConfig dir = do menv <- getMinimalEnvOverride case imgDockerBase dockerConfig of @@ -108,25 +133,26 @@ Just base -> do liftIO (writeFile - (toFilePath - (dir - $(mkRelFile "Dockerfile"))) + (toFilePath (dir $(mkRelFile "Dockerfile"))) (unlines ["FROM " ++ base, "ADD ./ /"])) - let args = [ "build" - , "-t" - , fromMaybe - (imageName (parent (parent (parent dir)))) - (imgDockerImageName dockerConfig) - , toFilePathNoTrailingSep dir] - callProcess $ Cmd Nothing "docker" menv args + let args = + [ "build" + , "-t" + , fromMaybe + (imageName (parent . parent . parent $ dir)) + (imgDockerImageName dockerConfig) + , toFilePathNoTrailingSep dir] + callProcess (Cmd Nothing "docker" menv args) --- | Extend the general purpose docker image with entrypoints (if --- specified). -extendDockerImageWithEntrypoint :: Assemble e m => ImageDockerOpts -> Path Abs Dir -> m () +-- | Extend the general purpose docker image with entrypoints (if specified). +extendDockerImageWithEntrypoint + :: Assemble e m + => ImageDockerOpts -> Path Abs Dir -> m () extendDockerImageWithEntrypoint dockerConfig dir = do menv <- getMinimalEnvOverride - let dockerImageName = fromMaybe - (imageName (parent (parent (parent dir)))) + let dockerImageName = + fromMaybe + (imageName (parent . parent . parent $ dir)) (imgDockerImageName dockerConfig) let imgEntrypoints = imgDockerEntrypoints dockerConfig case imgEntrypoints of @@ -134,46 +160,63 @@ Just eps -> forM_ eps - (\ep -> do - liftIO - (writeFile - (toFilePath - (dir - $(mkRelFile "Dockerfile"))) - (unlines - [ "FROM " ++ dockerImageName - , "ENTRYPOINT [\"/usr/local/bin/" ++ - ep ++ "\"]" - , "CMD []"])) - callProcess $ Cmd - Nothing - "docker" - menv - [ "build" - , "-t" - , dockerImageName ++ "-" ++ ep - , toFilePathNoTrailingSep dir]) + (\ep -> + do liftIO + (writeFile + (toFilePath + (dir $(mkRelFile "Dockerfile"))) + (unlines + [ "FROM " ++ dockerImageName + , "ENTRYPOINT [\"/usr/local/bin/" ++ + ep ++ "\"]" + , "CMD []"])) + callProcess + (Cmd + Nothing + "docker" + menv + [ "build" + , "-t" + , dockerImageName ++ "-" ++ ep + , toFilePathNoTrailingSep dir])) +-- | Fail with friendly error if project root not set. +fromMaybeProjectRoot :: Maybe (Path Abs Dir) -> Path Abs Dir +fromMaybeProjectRoot = + fromMaybe (throw StackImageCannotDetermineProjectRootException) + -- | The command name for dealing with images. -imgCmdName :: String +imgCmdName + :: String imgCmdName = "image" -- | The command name for building a docker container. -imgDockerCmdName :: String +imgDockerCmdName + :: String imgDockerCmdName = "container" -- | Convert image opts monoid to image options. -imgOptsFromMonoid :: ImageOptsMonoid -> ImageOpts -imgOptsFromMonoid ImageOptsMonoid{..} = ImageOpts +imgOptsFromMonoid + :: ImageOptsMonoid -> ImageOpts +imgOptsFromMonoid ImageOptsMonoid{..} = + ImageOpts { imgDockers = imgMonoidDockers } -- | Stack image exceptions. -data StackImageException = - StackImageDockerBaseUnspecifiedException +data StackImageException + = StackImageDockerBaseUnspecifiedException -- ^ Unspecified parent docker + -- container makes building + -- impossible + | StackImageCannotDetermineProjectRootException -- ^ Can't determine the + -- project root (where to + -- put image sandbox). deriving (Typeable) instance Exception StackImageException instance Show StackImageException where - show StackImageDockerBaseUnspecifiedException = "You must specify a base docker image on which to place your haskell executables." + show StackImageDockerBaseUnspecifiedException = + "You must specify a base docker image on which to place your haskell executables." + show StackImageCannotDetermineProjectRootException = + "Stack was unable to determine the project root in order to build a container." diff --git a/src/Stack/Init.hs b/src/Stack/Init.hs --- a/src/Stack/Init.hs +++ b/src/Stack/Init.hs @@ -22,18 +22,23 @@ import Data.Function (on) import qualified Data.HashMap.Strict as HM import qualified Data.IntMap as IntMap -import Data.List (intersect, maximumBy) -import Data.List.Extra (nubOrd) +import Data.List ( intercalate, intersect + , maximumBy) +import Data.List.NonEmpty (NonEmpty(..)) +import qualified Data.List.NonEmpty as NonEmpty import Data.Map (Map) import qualified Data.Map as Map -import Data.Maybe (fromJust) +import Data.Maybe import Data.Monoid import qualified Data.Text as T import qualified Data.Yaml as Yaml import qualified Distribution.PackageDescription as C +import qualified Distribution.Text as C +import qualified Distribution.Version as C import Network.HTTP.Client.Conduit (HasHttpManager) import Path import Path.IO +import qualified Paths_stack as Meta import Stack.BuildPlan import Stack.Config (getSnapshots, makeConcreteResolver) @@ -70,6 +75,7 @@ \file. Please try \"stack new\" instead." find = findCabalFiles (includeSubDirs initOpts) dirs' = if null dirs then [currDir] else dirs + $logInfo "Looking for .cabal or package.yaml files to use to init the project." cabalfps <- liftM concat $ mapM find dirs' (bundle, dupPkgs) <- cabalPackagesCheck cabalfps noPkgMsg Nothing @@ -79,23 +85,23 @@ let ignored = Map.difference bundle rbundle dupPkgMsg | (dupPkgs /= []) = - "Warning: Some packages were found to have names conflicting \ - \with others and have been commented out in the \ - \packages section.\n" + "Warning (added by new or init): Some packages were found to \ + \have names conflicting with others and have been commented \ + \out in the packages section.\n" | otherwise = "" missingPkgMsg | (Map.size ignored > 0) = - "Warning: Some packages were found to be incompatible with \ - \the resolver and have been left commented out in the \ - \packages section.\n" + "Warning (added by new or init): Some packages were found to \ + \be incompatible with the resolver and have been left commented \ + \out in the packages section.\n" | otherwise = "" extraDepMsg | (Map.size extraDeps > 0) = - "Warning: Specified resolver could not satisfy all \ - \dependencies. Some external packages have been added \ - \as dependencies.\n" + "Warning (added by new or init): Specified resolver could not \ + \satisfy all dependencies. Some external packages have been \ + \added as dependencies.\n" | otherwise = "" makeUserMsg msgs = @@ -129,8 +135,7 @@ pkgs = map toPkg $ Map.elems (fmap (parent . fst) rbundle) toPkg dir = PackageEntry - { peValidWanted = Nothing - , peExtraDepMaybe = Nothing + { peExtraDep = False , peLocation = PLFilePath $ makeRelDir dir , peSubdirs = [] } @@ -177,65 +182,129 @@ _ -> assert False $ B.byteString $ Yaml.encode p where renderObject o = - B.byteString "# This file was automatically generated by stack init\n" <> - B.byteString "# For more information, see: http://docs.haskellstack.org/en/stable/yaml_configuration/\n\n" <> - F.foldMap (goComment o) comments <> - goOthers (o `HM.difference` HM.fromList comments) <> - B.byteString - "# Control whether we use the GHC we find on the path\n\ - \# system-ghc: true\n\n\ - \# Require a specific version of stack, using version ranges\n\ - \# require-stack-version: -any # Default\n\ - \# require-stack-version: >= 1.0.0\n\n\ - \# Override the architecture used by stack, especially useful on Windows\n\ - \# arch: i386\n\ - \# arch: x86_64\n\n\ - \# Extra directories used by stack for building\n\ - \# extra-include-dirs: [/path/to/dir]\n\ - \# extra-lib-dirs: [/path/to/dir]\n\n\ - \# Allow a newer minor version of GHC than the snapshot specifies\n\ - \# compiler-check: newer-minor\n" + B.byteString headerHelp + <> B.byteString "\n\n" + <> F.foldMap (goComment o) comments + <> goOthers (o `HM.difference` HM.fromList comments) + <> B.byteString footerHelp - comments = - [ ("user-message", "A message to be displayed to the user. Used when autogenerated config ignored some packages or added extra deps.") - , ("resolver", "Specifies the GHC version and set of packages available (e.g., lts-3.5, nightly-2015-09-21, ghc-7.10.2)") - , ("packages", "Local packages, usually specified by relative directory name") - , ("extra-deps", "Packages to be pulled from upstream that are not in the resolver (e.g., acme-missiles-0.3)") - , ("flags", "Override default flag values for local packages and extra-deps") - , ("extra-package-dbs", "Extra package databases containing global packages") - ] + goComment o (name, comment) = + case HM.lookup name o of + Nothing -> assert (name == "user-message") mempty + Just v -> + B.byteString comment <> + B.byteString "\n" <> + B.byteString (Yaml.encode $ Yaml.object [(name, v)]) <> + if (name == "packages") then commentedPackages else "" <> + B.byteString "\n" + commentHelp = BC.pack . intercalate "\n" . map ("# " ++) commentedPackages = - let ignoredComment = "# The following packages have been ignored \ - \due to incompatibility with the resolver compiler or \ - \dependency conflicts with other packages" - dupComment = "# The following packages have been ignored due \ - \to package name conflict with other packages" + let ignoredComment = commentHelp + [ "The following packages have been ignored due to incompatibility with the" + , "resolver compiler, dependency conflicts with other packages" + , "or unsatisfied dependencies." + ] + dupComment = commentHelp + [ "The following packages have been ignored due to package name conflict " + , "with other packages." + ] in commentPackages ignoredComment ignoredPackages <> commentPackages dupComment dupPackages commentPackages comment pkgs | pkgs /= [] = - B.byteString (BC.pack $ comment ++ "\n") + B.byteString comment + <> B.byteString "\n" <> (B.byteString $ BC.pack $ concat $ (map (\x -> "#- " ++ x ++ "\n") pkgs) ++ ["\n"]) | otherwise = "" - goComment o (name, comment) = - case HM.lookup name o of - Nothing -> assert (name == "user-message") mempty - Just v -> - B.byteString "# " <> - B.byteString comment <> - B.byteString "\n" <> - B.byteString (Yaml.encode $ Yaml.object [(name, v)]) <> - if (name == "packages") then commentedPackages else "" <> - B.byteString "\n" - goOthers o | HM.null o = mempty | otherwise = assert False $ B.byteString $ Yaml.encode o + -- Per Section Help + comments = + [ ("user-message" , userMsgHelp) + , ("resolver" , resolverHelp) + , ("packages" , packageHelp) + , ("extra-deps" , "# Dependency packages to be pulled from upstream that are not in the resolver\n# (e.g., acme-missiles-0.3)") + , ("flags" , "# Override default flag values for local packages and extra-deps") + , ("extra-package-dbs", "# Extra package databases containing global packages") + ] + + -- Help strings + headerHelp = commentHelp + [ "This file was automatically generated by 'stack init'" + , "" + , "Some commonly used options have been documented as comments in this file." + , "For advanced use and comprehensive documentation of the format, please see:" + , "http://docs.haskellstack.org/en/stable/yaml_configuration/" + ] + + resolverHelp = commentHelp + [ "Resolver to choose a 'specific' stackage snapshot or a compiler version." + , "A snapshot resolver dictates the compiler version and the set of packages" + , "to be used for project dependencies. For example:" + , "" + , "resolver: lts-3.5" + , "resolver: nightly-2015-09-21" + , "resolver: ghc-7.10.2" + , "resolver: ghcjs-0.1.0_ghc-7.10.2" + , "resolver:" + , " name: custom-snapshot" + , " location: \"./custom-snapshot.yaml\"" + ] + + userMsgHelp = commentHelp + [ "A warning or info to be displayed to the user on config load." ] + + packageHelp = commentHelp + [ "User packages to be built." + , "Various formats can be used as shown in the example below." + , "" + , "packages:" + , "- some-directory" + , "- https://example.com/foo/bar/baz-0.0.2.tar.gz" + , "- location:" + , " git: https://github.com/commercialhaskell/stack.git" + , " commit: e7b331f14bcffb8367cd58fbfc8b40ec7642100a" + , "- location: https://github.com/commercialhaskell/stack/commit/e7b331f14bcffb8367cd58fbfc8b40ec7642100a" + , " extra-dep: true" + , " subdirs:" + , " - auto-update" + , " - wai" + , "" + , "A package marked 'extra-dep: true' will only be built if demanded by a" + , "non-dependency (i.e. a user package), and its test suites and benchmarks" + , "will not be run. This is useful for tweaking upstream packages." + ] + + footerHelp = + let major = toCabalVersion + $ toMajorVersion $ fromCabalVersion Meta.version + in commentHelp + [ "Control whether we use the GHC we find on the path" + , "system-ghc: true" + , "" + , "Require a specific version of stack, using version ranges" + , "require-stack-version: -any # Default" + , "require-stack-version: \"" + ++ C.display (C.orLaterVersion major) ++ "\"" + , "" + , "Override the architecture used by stack, especially useful on Windows" + , "arch: i386" + , "arch: x86_64" + , "" + , "Extra directories used by stack for building" + , "extra-include-dirs: [/path/to/dir]" + , "extra-lib-dirs: [/path/to/dir]" + , "" + , "Allow a newer minor version of GHC than the snapshot specifies" + , "compiler-check: newer-minor" + ] + getSnapshots' :: (MonadIO m, MonadMask m, MonadReader env m, HasConfig env, HasHttpManager env, MonadLogger m, MonadBaseControl IO m) => m Snapshots getSnapshots' = @@ -282,7 +351,7 @@ -- TODO support selecting best across regular and custom snapshots selectSnapResolver = do let gpds = Map.elems (fmap snd bundle) - snaps <- getSnapshots' >>= getRecommendedSnapshots + snaps <- fmap getRecommendedSnapshots getSnapshots' (s, r) <- selectBestSnapshot gpds snaps case r of BuildPlanCheckFail {} | not (omitPackages initOpts) @@ -355,23 +424,31 @@ result <- checkResolverSpec gpds Nothing resolver case result of BuildPlanCheckOk f -> return $ Right (f, Map.empty) - BuildPlanCheckPartial f _ + BuildPlanCheckPartial f e | needSolver resolver initOpts -> do - $logWarn $ "*** Resolver " <> resolverName resolver - <> " will need external packages: " - $logWarn $ indent $ T.pack $ show result + warnPartial result solve f + | omitPackages initOpts -> do + warnPartial result + $logWarn "*** Omitting packages with unsatisfied dependencies" + return $ Left $ failedUserPkgs e | otherwise -> throwM $ ResolverPartial resolver (show result) BuildPlanCheckFail _ e _ - | (omitPackages initOpts) -> do + | omitPackages initOpts -> do $logWarn $ "*** Resolver compiler mismatch: " <> resolverName resolver $logWarn $ indent $ T.pack $ show result - let failed = Map.unions (Map.elems (fmap deNeededBy e)) - return $ Left (Map.keys failed) + return $ Left $ failedUserPkgs e | otherwise -> throwM $ ResolverMismatch resolver (show result) where - indent t = T.unlines $ fmap (" " <>) (T.lines t) + indent t = T.unlines $ fmap (" " <>) (T.lines t) + warnPartial res = do + $logWarn $ "*** Resolver " <> resolverName resolver + <> " will need external packages: " + $logWarn $ indent $ T.pack $ show res + + failedUserPkgs e = Map.keys $ Map.unions (Map.elems (fmap deNeededBy e)) + gpds = Map.elems (fmap snd bundle) solve flags = do let cabalDirs = map parent (Map.elems (fmap fst bundle)) @@ -423,18 +500,17 @@ needSolver (ResolverCompiler _) _ = True needSolver _ _ = False -getRecommendedSnapshots :: (MonadIO m, MonadMask m, MonadReader env m, HasConfig env, HasHttpManager env, HasGHCVariant env, MonadLogger m, MonadBaseControl IO m) - => Snapshots - -> m [SnapName] -getRecommendedSnapshots snapshots = do +getRecommendedSnapshots :: Snapshots -> (NonEmpty SnapName) +getRecommendedSnapshots snapshots = -- in order - Latest LTS, Latest Nightly, all LTS most recent first - return $ nubOrd $ concat - [ map (uncurry LTS) - (take 1 $ reverse $ IntMap.toList $ snapshotsLts snapshots) - , [Nightly $ snapshotsNightly snapshots] - , map (uncurry LTS) - (drop 1 $ reverse $ IntMap.toList $ snapshotsLts snapshots) - ] + case NonEmpty.nonEmpty ltss of + Just (mostRecent :| older) + -> mostRecent :| (nightly : older) + Nothing + -> nightly :| [] + where + ltss = map (uncurry LTS) (IntMap.toDescList $ snapshotsLts snapshots) + nightly = Nightly (snapshotsNightly snapshots) data InitOpts = InitOpts { searchDirs :: ![T.Text] diff --git a/src/Stack/New.hs b/src/Stack/New.hs --- a/src/Stack/New.hs +++ b/src/Stack/New.hs @@ -1,5 +1,6 @@ {-# LANGUAGE ScopedTypeVariables #-} {-# LANGUAGE DeriveDataTypeable #-} +{-# LANGUAGE FlexibleContexts #-} {-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE TemplateHaskell #-} @@ -29,6 +30,7 @@ import qualified Data.ByteString.Lazy.Char8 as L8 import Data.Conduit import Data.Foldable (asum) +import qualified Data.HashMap.Strict as HM import Data.List import Data.Map.Strict (Map) import qualified Data.Map.Strict as M @@ -45,6 +47,7 @@ import Data.Time.Calendar import Data.Time.Clock import Data.Typeable +import qualified Data.Yaml as Yaml import Network.HTTP.Client.Conduit hiding (path) import Network.HTTP.Download import Network.HTTP.Types.Status @@ -57,6 +60,7 @@ import System.Process.Run import Text.Hastache import Text.Hastache.Context +import Text.Printf import Text.ProjectTemplate -------------------------------------------------------------------------------- @@ -205,9 +209,11 @@ throwM (InvalidTemplate template (show e))) when (M.null files) $ throwM (InvalidTemplate template "Template does not contain any files") - unless (any (".cabal" `isSuffixOf`) . M.keys $ files) $ - throwM (InvalidTemplate template "Template does not contain a .cabal\ - \ file") + + let isPkgSpec f = ".cabal" `isSuffixOf` f || f == "package.yaml" + unless (any isPkgSpec . M.keys $ files) $ + throwM (InvalidTemplate template "Template does not contain a .cabal \ + \or package.yaml file") liftM M.fromList (mapM @@ -262,15 +268,24 @@ (\(_ :: ProcessExitedUnsuccessfully) -> $logInfo "git init failed to run, ignoring ...") --------------------------------------------------------------------------------- --- Getting templates list - +-- | Display the set of templates accompanied with description if available. listTemplates :: (MonadIO m, MonadThrow m, MonadReader r m, HasHttpManager r, MonadCatch m, MonadLogger m) => m () listTemplates = do templates <- getTemplates - mapM_ (liftIO . T.putStrLn . templateName) (S.toList templates) + templateInfo <- getTemplateInfo + if not . M.null $ templateInfo then do + let keySizes = map (T.length . templateName) $ S.toList templates + padWidth = show $ maximum keySizes + outputfmt = "%-" <> padWidth <> "s %s\n" + headerfmt = "%-" <> padWidth <> "s %s\n" + liftIO $ printf headerfmt ("Template"::String) ("Description"::String) + forM_ (S.toList templates) (\x -> do + let name = templateName x + desc = fromMaybe "" $ liftM (mappend "- ") (M.lookup name templateInfo >>= description) + liftIO $ printf outputfmt (T.unpack name) (T.unpack desc)) + else mapM_ (liftIO . T.putStrLn . templateName) (S.toList templates) -- | Get the set of templates. getTemplates @@ -286,14 +301,37 @@ Left err -> throwM (BadTemplatesJSON err (responseBody resp)) Right value -> return value code -> throwM (BadTemplatesResponse code) + +getTemplateInfo + :: (MonadIO m, MonadThrow m, MonadReader r m, HasHttpManager r, MonadCatch m, MonadLogger m) + => m (Map Text TemplateInfo) +getTemplateInfo = do + req <- liftM addHeaders (parseUrl defaultTemplateInfoUrl) + resp <- catch (liftM Right $ httpLbs req) (\(ex :: HttpException) -> return . Left $ "Failed to download template info. The HTTP error was: " <> show ex) + case resp >>= is200 of + Left err -> do + liftIO . putStrLn $ err + return M.empty + Right resp' -> + case Yaml.decodeEither (LB.toStrict $ responseBody resp') :: Either String Object of + Left err -> + throwM $ BadTemplateInfo err + Right o -> + return (M.mapMaybe (Yaml.parseMaybe Yaml.parseJSON) (M.fromList . HM.toList $ o) :: Map Text TemplateInfo) where - addHeaders req = - req - { requestHeaders = [ ("User-Agent", "The Haskell Stack") - , ("Accept", "application/vnd.github.v3+json")] <> - requestHeaders req - } + is200 resp = + if statusCode (responseStatus resp) == 200 + then return resp + else Left $ "Unexpected status code while retrieving templates info: " <> show (statusCode $ responseStatus resp) +addHeaders :: Request -> Request +addHeaders req = + req + { requestHeaders = [ ("User-Agent", "The Haskell Stack") + , ("Accept", "application/vnd.github.v3+json")] <> + requestHeaders req + } + -- | Parser the set of templates from the JSON. parseTemplateSet :: Value -> Parser (Set TemplateName) parseTemplateSet a = do @@ -322,6 +360,11 @@ defaultTemplateUrl = "https://raw.githubusercontent.com/commercialhaskell/stack-templates/master" +-- | Default web URL to get a yaml file containing template metadata. +defaultTemplateInfoUrl :: String +defaultTemplateInfoUrl = + "https://raw.githubusercontent.com/commercialhaskell/stack-templates/master/template-info.yaml" + -- | Default web URL to list the repo contents. defaultTemplatesList :: String defaultTemplatesList = @@ -343,6 +386,9 @@ | MissingParameters !PackageName !TemplateName !(Set String) !(Path Abs File) | InvalidTemplate !TemplateName !String | AttemptedOverwrites [Path Abs File] + | FailedToDownloadTemplateInfo !HttpException + | BadTemplateInfo !String + | BadTemplateInfoResponse !Int deriving (Typeable) instance Exception NewException @@ -405,3 +451,9 @@ "The template would create the following files, but they already exist:\n" <> unlines (map ((" " ++) . toFilePath) fps) <> "Use --force to ignore this, and overwite these files." + show (FailedToDownloadTemplateInfo ex) = + "Failed to download templates info. The HTTP error was: " <> show ex + show (BadTemplateInfo err) = + "Template info couldn't be parsed: " <> err + show (BadTemplateInfoResponse code) = + "Unexpected status code while retrieving templates info: " <> show code diff --git a/src/Stack/Nix.hs b/src/Stack/Nix.hs --- a/src/Stack/Nix.hs +++ b/src/Stack/Nix.hs @@ -15,34 +15,33 @@ import Control.Arrow ((***)) import Control.Exception (Exception,throw) import Control.Monad hiding (mapM) -import Control.Monad.Catch (try,MonadCatch) +import Control.Monad.Catch (MonadMask) import Control.Monad.IO.Class (MonadIO,liftIO) import Control.Monad.Logger (MonadLogger,logDebug) import Control.Monad.Reader (MonadReader,asks) import Control.Monad.Trans.Control (MonadBaseControl) import Data.Char (toUpper) import Data.List (intercalate) -import Data.Traversable import Data.Maybe import Data.Monoid -import Data.Streaming.Process (ProcessExitedUnsuccessfully(..)) import qualified Data.Text as T -import Data.Version (showVersion) +import Data.Traversable import Data.Typeable (Typeable) +import Data.Version (showVersion) import Network.HTTP.Client.Conduit (HasHttpManager) import Path import Path.IO import qualified Paths_stack as Meta import Prelude hiding (mapM) -- Fix redundant import warnings -import Stack.Constants (stackProgName,platformVariantEnvVar) import Stack.Config (makeConcreteResolver) +import Stack.Config.Nix (nixCompiler) +import Stack.Constants (stackProgName,platformVariantEnvVar) import Stack.Docker (reExecArgName) import Stack.Exec (exec) -import System.Process.Read (getEnvOverride) import Stack.Types import Stack.Types.Internal import System.Environment (lookupEnv,getArgs,getExecutablePath) -import System.Exit (exitSuccess, exitWith) +import System.Process.Read (getEnvOverride) -- | If Nix is enabled, re-runs the currently running OS command in a Nix container. -- Otherwise, runs the inner action. @@ -85,7 +84,7 @@ traverse (resolveFile (fromMaybeProjectRoot mprojectRoot)) $ nixInitFile (configNix config) let pkgsInConfig = nixPackages (configNix config) - pkgs = pkgsInConfig ++ [nixCompiler (configNix config) mresolver mcompiler] + pkgs = pkgsInConfig ++ [nixCompiler config mresolver mcompiler] pureShell = nixPureShell (configNix config) nixopts = case mshellFile of Just fp -> [toFilePath fp] @@ -105,17 +104,15 @@ fullArgs = concat [if pureShell then ["--pure"] else [], map T.unpack (nixShellOptions (configNix config)) ,nixopts - ,["--command", intercalate " " (cmnd:"$STACK_IN_NIX_EXTRA_ARGS":args)] + ,["--run", intercalate " " (cmnd:"$STACK_IN_NIX_EXTRA_ARGS":args)] ] - + -- Using --run instead of --command so we cannot + -- end up in the nix-shell if stack build is Ctrl-C'd $logDebug $ "Using a nix-shell environment " <> (case mshellFile of Just path -> "from file: " <> (T.pack (toFilePath path)) Nothing -> "with nix packages: " <> (T.intercalate ", " pkgs)) - e <- try (exec envOverride "nix-shell" fullArgs) - case e of - Left (ProcessExitedUnsuccessfully _ ec) -> liftIO (exitWith ec) - Right () -> liftIO exitSuccess + exec envOverride "nix-shell" fullArgs -- | Shell-escape quotes inside the string and enclose it in quotes. escape :: String -> String @@ -162,7 +159,7 @@ ,MonadReader env m ,MonadLogger m ,MonadBaseControl IO m - ,MonadCatch m + ,MonadMask m ,HasConfig env ,HasTerminal env ,HasReExec env diff --git a/src/Stack/Options.hs b/src/Stack/Options.hs --- a/src/Stack/Options.hs +++ b/src/Stack/Options.hs @@ -25,10 +25,11 @@ ,hpcReportOptsParser ,pvpBoundsOption ,globalOptsFromMonoid + ,splitObjsWarning ) where import Control.Monad.Logger (LogLevel (..)) -import Data.Char (isSpace, toLower) +import Data.Char (isSpace, toLower, toUpper) import Data.List (intercalate) import Data.List.Split (splitOn) import qualified Data.Map as Map @@ -44,10 +45,12 @@ import Options.Applicative.Args import Options.Applicative.Builder.Extra import Options.Applicative.Types (fromM, oneM, readerAsk) +import Path +import Stack.Build (splitObjsWarning) import Stack.Clean (CleanOpts (..)) import Stack.Config (packagesParser) import Stack.ConfigCmd -import Stack.Constants (stackProgName) +import Stack.Constants import Stack.Coverage (HpcReportOpts (..)) import Stack.Docker import qualified Stack.Docker as Docker @@ -59,15 +62,6 @@ import Stack.Types import Stack.Types.TemplateName --- | Command sum type for conditional arguments. -data BuildCommand - = Build - | Test - | Haddock - | Bench - | Install - deriving (Eq) - -- | Allows adjust global options depending on their context -- Note: This was being used to remove ambibuity between the local and global -- implementation of stack init --resolver option. Now that stack init has no @@ -76,177 +70,96 @@ data GlobalOptsContext = OuterGlobalOpts -- ^ Global options before subcommand name | OtherCmdGlobalOpts -- ^ Global options following any other subcommand + | BuildCmdGlobalOpts deriving (Show, Eq) -- | Parser for bench arguments. -benchOptsParser :: Parser BenchmarkOpts -benchOptsParser = BenchmarkOpts +-- FIXME hiding options +benchOptsParser :: Bool -> Parser BenchmarkOptsMonoid +benchOptsParser hide0 = BenchmarkOptsMonoid <$> optional (strOption (long "benchmark-arguments" <> metavar "BENCH_ARGS" <> help ("Forward BENCH_ARGS to the benchmark suite. " <> - "Supports templates from `cabal bench`"))) - <*> switch (long "no-run-benchmarks" <> - help "Disable running of benchmarks. (Benchmarks will still be built.)") + "Supports templates from `cabal bench`") <> + hide)) + <*> optional (switch (long "no-run-benchmarks" <> + help "Disable running of benchmarks. (Benchmarks will still be built.)" <> + hide)) + where hide = hideMods hide0 --- | Parser for build arguments. +-- | Parser for CLI-only build arguments buildOptsParser :: BuildCommand - -> Parser BuildOpts + -> Parser BuildOptsCLI buildOptsParser cmd = - transform <$> trace <*> profile <*> options - where transform tracing profiling = enable - where enable opts - | tracing || profiling = - opts {boptsLibProfile = True - ,boptsExeProfile = True - ,boptsGhcOptions = ["-auto-all","-caf-all"] - ,boptsBenchmarkOpts = - bopts {beoAdditionalArgs = - beoAdditionalArgs bopts <> - Just (" " <> unwords additionalArgs)} - ,boptsTestOpts = - topts {toAdditionalArgs = - (toAdditionalArgs topts) <> - additionalArgs}} - | otherwise = opts - where bopts = boptsBenchmarkOpts opts - topts = boptsTestOpts opts - additionalArgs = "+RTS" : catMaybes [trac, prof, Just "-RTS"] - trac = if tracing - then Just "-xc" - else Nothing - prof = if profiling - then Just "-p" - else Nothing - profile = - flag False True - ( long "profile" - <> help "Enable profiling in libraries, executables, etc. \ - \for all expressions and generate a profiling report\ - \ in exec or benchmarks") - trace = - flag False True - ( long "trace" - <> help "Enable profiling in libraries, executables, etc. \ - \for all expressions and generate a backtrace on \ - \exception") - options = - BuildOpts <$> target <*> libProfiling <*> exeProfiling <*> - haddock <*> haddockDeps <*> dryRun <*> ghcOpts <*> - flags <*> copyBins <*> preFetch <*> buildSubset <*> - fileWatch' <*> keepGoing <*> forceDirty <*> tests <*> - testOptsParser <*> benches <*> benchOptsParser <*> - many exec <*> onlyConfigure <*> reconfigure <*> cabalVerbose - target = - many (textArgument - (metavar "TARGET" <> - help "If none specified, use all packages")) - libProfiling = - boolFlags False - "library-profiling" - "library profiling for TARGETs and all its dependencies" - idm - exeProfiling = - boolFlags False - "executable-profiling" - "executable profiling for TARGETs and all its dependencies" - idm - haddock = - boolFlags (cmd == Haddock) - "haddock" - "generating Haddocks the package(s) in this directory/configuration" - idm - haddockDeps = - maybeBoolFlags - "haddock-deps" - "building Haddocks for dependencies" - idm - copyBins = boolFlags (cmd == Install) - "copy-bins" - "copying binaries to the local-bin-path (see 'stack path')" - idm - - dryRun = switch (long "dry-run" <> - help "Don't build anything, just prepare to") - ghcOpts = (\x y z -> concat [x, y, z]) - <$> flag [] ["-Wall", "-Werror"] - ( long "pedantic" - <> help "Turn on -Wall and -Werror" - ) - <*> flag [] ["-O0"] - ( long "fast" - <> help "Turn off optimizations (-O0)" - ) - <*> many (textOption (long "ghc-options" <> - metavar "OPTION" <> - help "Additional options passed to GHC")) - - flags = Map.unionsWith Map.union <$> many - (option readFlag - (long "flag" <> - metavar "PACKAGE:[-]FLAG" <> - help ("Override flags set in stack.yaml " <> - "(applies to local packages and extra-deps)"))) - - preFetch = switch - (long "prefetch" <> - help "Fetch packages necessary for the build immediately, useful with --dry-run") - - buildSubset = - flag' BSOnlyDependencies - (long "dependencies-only" <> - help "A synonym for --only-dependencies") - <|> flag' BSOnlySnapshot - (long "only-snapshot" <> - help "Only build packages for the snapshot database, not the local database") - <|> flag' BSOnlyDependencies - (long "only-dependencies" <> - help "Only build packages that are dependencies of targets on the command line") - <|> pure BSAll - - fileWatch' = - flag' FileWatch - (long "file-watch" <> - help "Watch for changes in local files and automatically rebuild. Ignores files in VCS boring/ignore file") - <|> flag' FileWatchPoll - (long "file-watch-poll" <> - help "Like --file-watch, but polling the filesystem instead of using events") - <|> pure NoFileWatch - - keepGoing = maybeBoolFlags - "keep-going" - "continue running after a step fails (default: false for build, true for test/bench)" - idm - - forceDirty = switch - (long "force-dirty" <> - help "Force treating all local packages as having dirty files (useful for cases where stack can't detect a file change)") - - tests = boolFlags (cmd == Test) - "test" - "testing the package(s) in this directory/configuration" - idm - - benches = boolFlags (cmd == Bench) - "bench" - "benchmarking the package(s) in this directory/configuration" - idm - - exec = cmdOption - ( long "exec" <> + BuildOptsCLI <$> + many + (textArgument + (metavar "TARGET" <> + help "If none specified, use all packages")) <*> + switch + (long "dry-run" <> + help "Don't build anything, just prepare to") <*> + ((\x y z -> + concat [x, y, z]) <$> + flag + [] + ["-Wall", "-Werror"] + (long "pedantic" <> + help "Turn on -Wall and -Werror") <*> + flag + [] + ["-O0"] + (long "fast" <> + help "Turn off optimizations (-O0)") <*> + many + (textOption + (long "ghc-options" <> + metavar "OPTION" <> + help "Additional options passed to GHC"))) <*> + (Map.unionsWith Map.union <$> + many + (option + readFlag + (long "flag" <> + metavar "PACKAGE:[-]FLAG" <> + help + ("Override flags set in stack.yaml " <> + "(applies to local packages and extra-deps)")))) <*> + (flag' + BSOnlyDependencies + (long "dependencies-only" <> + help "A synonym for --only-dependencies") <|> + flag' + BSOnlySnapshot + (long "only-snapshot" <> + help + "Only build packages for the snapshot database, not the local database") <|> + flag' + BSOnlyDependencies + (long "only-dependencies" <> + help + "Only build packages that are dependencies of targets on the command line") <|> + pure BSAll) <*> + (flag' + FileWatch + (long "file-watch" <> + help + "Watch for changes in local files and automatically rebuild. Ignores files in VCS boring/ignore file") <|> + flag' + FileWatchPoll + (long "file-watch-poll" <> + help + "Like --file-watch, but polling the filesystem instead of using events") <|> + pure NoFileWatch) <*> + many (cmdOption + (long "exec" <> metavar "CMD [ARGS]" <> - help "Command and arguments to run after a successful build" ) - - onlyConfigure = switch - (long "only-configure" <> - help "Only perform the configure step, not any builds. Intended for tool usage, may break when used on multiple packages at once!") - - reconfigure = switch - (long "reconfigure" <> - help "Perform the configure step even if unnecessary. Useful in some corner cases with custom Setup.hs files") - - cabalVerbose = switch - (long "cabal-verbose" <> - help "Ask Cabal to be verbose in its output") + help "Command and arguments to run after a successful build")) <*> + switch + (long "only-configure" <> + help + "Only perform the configure step, not any builds. Intended for tool usage, may break when used on multiple packages at once!") <*> + pure cmd -- | Parser for package:[-]flag readFlag :: ReadM (Map (Maybe PackageName) (Map FlagName Bool)) @@ -273,7 +186,7 @@ -- | Command-line parser for the clean command. cleanOptsParser :: Parser CleanOpts -cleanOptsParser = CleanTargets <$> packages <|> CleanFull <$> doFullClean +cleanOptsParser = CleanShallow <$> packages <|> doFullClean where packages = many @@ -281,15 +194,18 @@ (metavar "PACKAGE" <> help "If none specified, clean all local packages")) doFullClean = - switch + flag' + CleanFull (long "full" <> - help "Remove whole the work dir, default is .stack-work") + help "Delete all work directories (.stack-work by default) in the project") -- | Command-line arguments parser for configuration. -configOptsParser :: Bool -> Parser ConfigMonoid +configOptsParser :: GlobalOptsContext -> Parser ConfigMonoid configOptsParser hide0 = - (\workDir dockerOpts nixOpts systemGHC installGHC arch os ghcVariant jobs includes libs skipGHCCheck skipMsys localBin modifyCodePage allowDifferentUser -> mempty - { configMonoidWorkDir = workDir + (\stackRoot workDir buildOpts dockerOpts nixOpts systemGHC installGHC arch os ghcVariant jobs includes libs skipGHCCheck skipMsys localBin modifyCodePage allowDifferentUser -> mempty + { configMonoidStackRoot = stackRoot + , configMonoidWorkDir = workDir + , configMonoidBuildOpts = buildOpts , configMonoidDockerOpts = dockerOpts , configMonoidNixOpts = nixOpts , configMonoidSystemGHC = systemGHC @@ -306,12 +222,20 @@ , configMonoidModifyCodePage = modifyCodePage , configMonoidAllowDifferentUser = allowDifferentUser }) - <$> optional (strOption + <$> optional (option readAbsDir + ( long stackRootOptionName + <> metavar (map toUpper stackRootOptionName) + <> help ("Absolute path to the global stack root directory " ++ + "(Overrides any STACK_ROOT environment variable)") + <> hide + )) + <*> optional (strOption ( long "work-dir" <> metavar "WORK-DIR" <> help "Override work directory (default: .stack-work)" <> hide )) + <*> buildOptsMonoidParser (hide0 /= BuildCmdGlobalOpts) <*> dockerOptsParser True <*> nixOptsParser True <*> maybeBoolFlags @@ -334,7 +258,7 @@ <> help "Operating system, e.g. linux, windows" <> hide )) - <*> optional (ghcVariantParser hide0) + <*> optional (ghcVariantParser (hide0 /= OuterGlobalOpts)) <*> optional (option auto ( long "jobs" <> short 'j' @@ -377,8 +301,151 @@ ("permission for users other than the owner of the stack root " ++ "directory to use a stack installation (POSIX only)") hide - where hide = hideMods hide0 + where hide = hideMods (hide0 /= OuterGlobalOpts) +readAbsDir :: ReadM (Path Abs Dir) +readAbsDir = do + s <- readerAsk + case parseAbsDir s of + Just p -> return p + Nothing -> + readerError + ("Failed to parse absolute path to directory: '" ++ s ++ "'") + +buildOptsMonoidParser :: Bool -> Parser BuildOptsMonoid +buildOptsMonoidParser hide0 = + transform <$> trace <*> profile <*> options + where + hide = + hideMods hide0 + transform tracing profiling = + enable + where + enable opts + | tracing || profiling = + opts + { buildMonoidLibProfile = Just True + , buildMonoidExeProfile = Just True + , buildMonoidBenchmarkOpts = bopts + { beoMonoidAdditionalArgs = beoMonoidAdditionalArgs bopts <> + Just (" " <> unwords additionalArgs) + } + , buildMonoidTestOpts = topts + { toMonoidAdditionalArgs = (toMonoidAdditionalArgs topts) <> + additionalArgs + } + } + | otherwise = + opts + where + bopts = + buildMonoidBenchmarkOpts opts + topts = + buildMonoidTestOpts opts + additionalArgs = + "+RTS" : catMaybes [trac, prof, Just "-RTS"] + trac = + if tracing + then Just "-xc" + else Nothing + prof = + if profiling + then Just "-p" + else Nothing + profile = + flag + False + True + (long "profile" <> + help + "Enable profiling in libraries, executables, etc. \ + \for all expressions and generate a profiling report\ + \ in exec or benchmarks" <> + hide) + + trace = + flag + False + True + (long "trace" <> + help + "Enable profiling in libraries, executables, etc. \ + \for all expressions and generate a backtrace on \ + \exception" <> + hide) + options = + BuildOptsMonoid <$> libProfiling <*> exeProfiling <*> haddock <*> openHaddocks <*> + haddockDeps <*> copyBins <*> preFetch <*> keepGoing <*> forceDirty <*> + tests <*> testOptsParser hide0 <*> benches <*> benchOptsParser hide0 <*> reconfigure <*> + cabalVerbose <*> splitObjs + libProfiling = + maybeBoolFlags + "library-profiling" + "library profiling for TARGETs and all its dependencies" + hide + exeProfiling = + maybeBoolFlags + "executable-profiling" + "executable profiling for TARGETs and all its dependencies" + hide + haddock = + maybeBoolFlags + "haddock" + "generating Haddocks the package(s) in this directory/configuration" + hide + openHaddocks = + maybeBoolFlags + "open" + "opening the local Haddock documentation in the browser" + hide + haddockDeps = + maybeBoolFlags "haddock-deps" "building Haddocks for dependencies" hide + copyBins = + maybeBoolFlags + "copy-bins" + "copying binaries to the local-bin-path (see 'stack path')" + hide + keepGoing = + maybeBoolFlags + "keep-going" + "continue running after a step fails (default: false for build, true for test/bench)" + hide + preFetch = + maybeBoolFlags + "prefetch" + "Fetch packages necessary for the build immediately, useful with --dry-run" + hide + forceDirty = + maybeBoolFlags + "force-dirty" + "Force treating all local packages as having dirty files (useful for cases where stack can't detect a file change" + hide + tests = + maybeBoolFlags + "test" + "testing the package(s) in this directory/configuration" + hide + benches = + maybeBoolFlags + "bench" + "benchmarking the package(s) in this directory/configuration" + hide + reconfigure = + maybeBoolFlags + "reconfigure" + "Perform the configure step even if unnecessary. Useful in some corner cases with custom Setup.hs files" + hide + cabalVerbose = + maybeBoolFlags + "cabal-verbose" + "Ask Cabal to be verbose in its output" + hide + splitObjs = + maybeBoolFlags + "split-objs" + ("Enable split-objs, to reduce output size (at the cost of build time). " ++ splitObjsWarning) + hide + nixOptsParser :: Bool -> Parser NixOptsMonoid nixOptsParser hide0 = overrideActivation <$> (NixOptsMonoid @@ -651,7 +718,7 @@ optional (strOption (long Docker.reExecArgName <> hidden <> internal)) <*> optional (option auto (long dockerEntrypointArgName <> hidden <> internal)) <*> logLevelOptsParser hide0 defLogLevel <*> - configOptsParser hide0 <*> + configOptsParser kind <*> optional (abstractResolverOptsParser hide0) <*> optional (compilerOptsParser hide0) <*> maybeBoolFlags @@ -710,8 +777,13 @@ (short 'v' <> long "verbose" <> help ("Enable verbose mode: verbosity level \"" <> showLevel verboseLevel <> "\"") <> hideMods hide) <|> + flag' (Just silentLevel) + (long "silent" <> + help ("Enable silent mode: verbosity level \"" <> showLevel silentLevel <> "\"") <> + hideMods hide) <|> pure defLogLevel where verboseLevel = LevelDebug + silentLevel = LevelOther "silent" showLevel l = case l of LevelDebug -> "debug" @@ -790,20 +862,25 @@ idm -- | Parser for test arguments. -testOptsParser :: Parser TestOpts -testOptsParser = TestOpts - <$> boolFlags True - "rerun-tests" - "running already successful tests" - idm +-- FIXME hide args +testOptsParser :: Bool -> Parser TestOptsMonoid +testOptsParser hide0 = TestOptsMonoid + <$> maybeBoolFlags + "rerun-tests" + "running already successful tests" + hide <*> fmap (fromMaybe []) (optional (argsOption(long "test-arguments" <> metavar "TEST_ARGS" <> - help "Arguments passed in to the test suite program"))) - <*> switch (long "coverage" <> - help "Generate a code coverage report") - <*> switch (long "no-run-tests" <> - help "Disable running of tests. (Tests will still be built.)") + help "Arguments passed in to the test suite program" + <> hide))) + <*> optional (switch (long "coverage" <> + help "Generate a code coverage report" + <> hide)) + <*> optional (switch (long "no-run-tests" <> + help "Disable running of tests. (Tests will still be built.)" + <> hide)) + where hide = hideMods hide0 -- | Parser for @stack new@. newOptsParser :: Parser (NewOpts,InitOpts) diff --git a/src/Stack/Package.hs b/src/Stack/Package.hs --- a/src/Stack/Package.hs +++ b/src/Stack/Package.hs @@ -21,6 +21,7 @@ ,readPackageUnresolvedBS ,resolvePackage ,findOrGenerateCabalFile + ,hpack ,Package(..) ,GetPackageFiles(..) ,GetPackageOpts(..) @@ -67,10 +68,10 @@ import Distribution.Compiler import Distribution.ModuleName (ModuleName) import qualified Distribution.ModuleName as Cabal -import Distribution.Package hiding (Package,PackageName,packageName,packageVersion,PackageIdentifier) import qualified Distribution.Package as D -import Distribution.PackageDescription hiding (FlagName) +import Distribution.Package hiding (Package,PackageName,packageName,packageVersion,PackageIdentifier) import qualified Distribution.PackageDescription as D +import Distribution.PackageDescription hiding (FlagName) import Distribution.PackageDescription.Parse import qualified Distribution.PackageDescription.Parse as D import Distribution.ParseUtils @@ -78,6 +79,8 @@ import Distribution.System (OS (..), Arch, Platform (..)) import Distribution.Text (display, simpleParse) import qualified Distribution.Verbosity as D +import qualified Hpack +import qualified Hpack.Config as Hpack import Path as FL import Path.Extra import Path.Find @@ -91,8 +94,6 @@ import System.FilePath (splitExtensions, replaceExtension) import qualified System.FilePath as FilePath import System.IO.Error -import qualified Hpack -import qualified Hpack.Config as Hpack -- | Read the raw, unresolved package information. readPackageUnresolved :: (MonadIO m, MonadThrow m) @@ -187,6 +188,8 @@ , packageFiles = pkgFiles , packageTools = packageDescTools pkg , packageFlags = packageConfigFlags packageConfig + , packageDefaultFlags = M.fromList + [(fromCabalFlagName (flagName flag), flagDefault flag) | flag <- genPackageFlags gpkg] , packageAllDeps = S.fromList (M.keys deps) , packageHasLibrary = maybe False (buildable . libBuildInfo) (library pkg) , packageTests = M.fromList @@ -209,8 +212,6 @@ (not . null . exposedModules) (library pkg) , packageSimpleType = buildType (packageDescription gpkg) == Just Simple - , packageDefinedFlags = S.fromList $ - map (fromCabalFlagName . flagName) $ genPackageFlags gpkg } where pkgFiles = GetPackageFiles $ @@ -221,7 +222,7 @@ runReaderT (packageDescModulesAndFiles pkg) (cabalfp, buildDir distDir) - buildFiles <- liftM (S.insert cabalfp) $ + setupFiles <- if buildType pkg `elem` [Nothing, Just Custom] then do let setupHsPath = pkgDir $(mkRelFile "Setup.hs") @@ -231,6 +232,10 @@ setupLhsExists <- doesFileExist setupLhsPath if setupLhsExists then return (S.singleton setupLhsPath) else return S.empty else return S.empty + buildFiles <- liftM (S.insert cabalfp . S.union setupFiles) $ do + let hpackPath = pkgDir $(mkRelFile Hpack.packageConfig) + hpackExists <- doesFileExist hpackPath + return $ if hpackExists then S.singleton hpackPath else S.empty return (componentModules, componentFiles, buildFiles <> dataFiles', warnings) pkgId = package (packageDescription gpkg) name = fromCabalPackageName (pkgName pkgId) @@ -838,32 +843,41 @@ => Maybe String -- ^ Package component name -> [Path Abs Dir] -- ^ Directories to look in. -> [DotCabalDescriptor] -- ^ Base names. - -> [Text] -- ^ Extentions. + -> [Text] -- ^ Extensions. -> m (Set ModuleName,Set DotCabalPath,[PackageWarning]) resolveFilesAndDeps component dirs names0 exts = do - (dotCabalPaths,foundModules) <- loop names0 S.empty - warnings <- warnUnlisted foundModules + (dotCabalPaths, foundModules, missingModules) <- loop names0 S.empty + warnings <- liftM2 (++) (warnUnlisted foundModules) (warnMissing missingModules) return (foundModules, dotCabalPaths, warnings) where - loop [] doneModules = return (S.empty, doneModules) + loop [] _ = return (S.empty, S.empty, []) loop names doneModules0 = do - resolvedFiles <- resolveFiles dirs names exts - pairs <- mapM (getDependencies component) resolvedFiles - let doneModules' = + resolved <- resolveFiles dirs names exts + let foundFiles = mapMaybe snd resolved + (foundModules', missingModules') = partition (isJust . snd) resolved + foundModules = mapMaybe (dotCabalModule . fst) foundModules' + missingModules = mapMaybe (dotCabalModule . fst) missingModules' + pairs <- mapM (getDependencies component) foundFiles + let doneModules = S.union doneModules0 (S.fromList (mapMaybe dotCabalModule names)) moduleDeps = S.unions (map fst pairs) thDepFiles = concatMap snd pairs - modulesRemaining = S.difference moduleDeps doneModules' - (resolvedFiles',doneModules'') <- - loop (map DotCabalModule (S.toList modulesRemaining)) doneModules' + modulesRemaining = S.difference moduleDeps doneModules + -- Ignore missing modules discovered as dependencies - they may + -- have been deleted. + (resolvedFiles, resolvedModules, _) <- + loop (map DotCabalModule (S.toList modulesRemaining)) doneModules return ( S.union (S.fromList - (resolvedFiles <> map DotCabalFilePath thDepFiles)) - resolvedFiles' - , doneModules'') + (foundFiles <> map DotCabalFilePath thDepFiles)) + resolvedFiles + , S.union + (S.fromList foundModules) + resolvedModules + , missingModules) warnUnlisted foundModules = do let unlistedModules = foundModules `S.difference` @@ -876,7 +890,20 @@ cabalfp component (S.toList unlistedModules)] + warnMissing _missingModules = do + return [] + {- FIXME: the issue with this is it's noisy for modules like Paths_* + cabalfp <- asks fst + return $ + if null missingModules + then [] + else [ MissingModulesWarning + cabalfp + component + missingModules] + -} + -- | Get the dependencies of a Haskell module file. getDependencies :: (MonadReader (Path Abs File, Path Abs Dir) m, MonadIO m) @@ -945,10 +972,10 @@ :: (MonadIO m, MonadLogger m, MonadThrow m, MonadReader (Path Abs File, Path Abs Dir) m) => [Path Abs Dir] -- ^ Directories to look in. -> [DotCabalDescriptor] -- ^ Base names. - -> [Text] -- ^ Extentions. - -> m [DotCabalPath] + -> [Text] -- ^ Extensions. + -> m [(DotCabalDescriptor, Maybe DotCabalPath)] resolveFiles dirs names exts = - forMaybeM names (findCandidate dirs exts) + forM names (\name -> liftM (name, ) (findCandidate dirs exts name)) -- | Find a candidate for the given module-or-filename from the list -- of directories and given extensions. @@ -1066,21 +1093,54 @@ -- If the directory contains a file named package.yaml, hpack is used to -- generate a .cabal file from it. findOrGenerateCabalFile - :: (MonadThrow m, MonadIO m) + :: forall m. (MonadThrow m, MonadIO m) => Path Abs Dir -- ^ package directory -> m (Path Abs File) findOrGenerateCabalFile pkgDir = do - liftIO $ hpack pkgDir - files <- liftIO $ findFiles - pkgDir - (flip hasExtension "cabal" . FL.toFilePath) - (const False) - case files of - [] -> throwM $ PackageNoCabalFileFound pkgDir - [x] -> return x - _:_ -> throwM $ PackageMultipleCabalFilesFound pkgDir files - where hasExtension fp x = FilePath.takeExtension fp == "." ++ x + mPackageYaml <- Path.IO.findFile [pkgDir] $(mkRelFile "package.yaml") + case mPackageYaml of + Nothing -> findCabalFile + Just packageYaml -> do + eCabalFile <- findCabalFile' + case eCabalFile of + -- Check that cabal file is fresh enough + Right cabalFile -> do + packageYamlModified <- getModificationTime packageYaml + cabalFileModified <- getModificationTime cabalFile + when (cabalFileModified < packageYamlModified) $ liftIO $ hpack pkgDir + -- Important to research again + findCabalFile + -- If we cannot find cabal file, but have package.yaml + -- run hpack and try to find cabal file again + Left (PackageNoCabalFileFound _) -> do + liftIO $ hpack pkgDir + findCabalFile + + -- Rethrow exception + Left e -> throwM e + where + findCabalFile :: m (Path Abs File) + findCabalFile = findCabalFile' >>= either throwM return + + findCabalFile' :: m (Either PackageException (Path Abs File)) + findCabalFile' = do + files <- liftIO $ findFiles + pkgDir + (flip hasExtension "cabal" . FL.toFilePath) + (const False) + return $ case files of + [] -> Left $ PackageNoCabalFileFound pkgDir + [x] -> Right x + -- If there are multiple files, ignore files that start with + -- ".". On unixlike environments these are hidden, and this + -- character is not valid in package names. The main goal is + -- to ignore emacs lock files - see + -- https://github.com/commercialhaskell/stack/issues/1897. + (filter (not . ("." `isPrefixOf`) . toFilePath . filename) -> [x]) -> Right x + _:_ -> Left $ PackageMultipleCabalFilesFound pkgDir files + where hasExtension fp x = FilePath.takeExtension fp == "." ++ x + -- | Generate .cabal file from package.yaml, if necessary. hpack :: Path Abs Dir -> IO () hpack pkgDir = do @@ -1093,7 +1153,7 @@ => Package -> Maybe String -> m (Path Abs File) buildLogPath package' msuffix = do env <- ask - let stack = configProjectWorkDir env + let stack = getProjectWorkDir env fp <- parseRelFile $ concat $ packageIdentifierString (packageIdentifier package') : maybe id (\suffix -> ("-" :) . (suffix :)) msuffix [".log"] @@ -1123,7 +1183,7 @@ => FilePath.FilePath -> m (Maybe (Path Abs File)) resolveFileOrWarn = resolveOrWarn "File" f - where f p x = forgivingAbsence (resolveFile p x) + where f p x = forgivingAbsence (resolveFile p x) >>= rejectMissingFile -- | Resolve the directory, if it can't be resolved, warn for the user -- (purely to be helpful). @@ -1131,7 +1191,7 @@ => FilePath.FilePath -> m (Maybe (Path Abs Dir)) resolveDirOrWarn = resolveOrWarn "Directory" f - where f p x = forgivingAbsence (resolveDir p x) + where f p x = forgivingAbsence (resolveDir p x) >>= rejectMissingDir -- | Extract the @PackageIdentifier@ given an exploded haskell package -- path. diff --git a/src/Stack/PackageIndex.hs b/src/Stack/PackageIndex.hs --- a/src/Stack/PackageIndex.hs +++ b/src/Stack/PackageIndex.hs @@ -19,12 +19,16 @@ module Stack.PackageIndex ( updateAllIndices , getPackageCaches + , getPackageCachesIO + , getPackageVersions + , getPackageVersionsIO + , lookupPackageVersions ) where import qualified Codec.Archive.Tar as Tar import Control.Exception (Exception) import Control.Exception.Enclosed (tryIO) -import Control.Monad (unless, when, liftM) +import Control.Monad (unless, when, liftM, void) import Control.Monad.Catch (MonadThrow, throwM, MonadCatch) import qualified Control.Monad.Catch as C import Control.Monad.IO.Class (MonadIO, liftIO) @@ -41,9 +45,12 @@ sourceHandle) import Data.Conduit.Zlib (ungzip) import Data.Foldable (forM_) +import Data.IORef import Data.Int (Int64) import Data.Map (Map) import qualified Data.Map.Strict as Map +import Data.Set (Set) +import qualified Data.Set as Set import Data.Monoid import Data.Text (Text) import qualified Data.Text as T @@ -64,8 +71,11 @@ import System.FilePath (takeBaseName, (<.>)) import System.IO (IOMode (ReadMode, WriteMode), withBinaryFile) -import System.Process.Read (readInNull, readProcessNull, ReadProcessException(..), - EnvOverride, doesExecutableExist) +import System.Process.Read (EnvOverride, + ReadProcessException (..), + doesExecutableExist, readInNull, + tryProcessStdout) +import System.Process.Run (Cmd(..), callProcessInheritStderrStdout) -- | Populate the package index caches and return them. populateCache @@ -193,7 +203,8 @@ ,HasConfig env,MonadBaseControl IO m, MonadCatch m) => EnvOverride -> m () -updateAllIndices menv = +updateAllIndices menv = do + clearPackageCaches asks (configPackageIndices . getConfig) >>= mapM_ (updateIndex menv) -- | Update the index tarball @@ -244,42 +255,52 @@ unless repoExists (readInNull suDir "git" menv cloneArgs Nothing) $logSticky "Fetching package index ..." - readProcessNull (Just acfDir) menv "git" ["fetch","--tags","--depth=1"] `C.catch` \(ex :: ReadProcessException) -> do - -- we failed, so wipe the directory and try again, see #1418 - $logWarn (T.pack (show ex)) - $logStickyDone "Failed to fetch package index, retrying." - removeDirRecur acfDir - readInNull suDir "git" menv cloneArgs Nothing - $logSticky "Fetching package index ..." - readInNull acfDir "git" menv ["fetch","--tags","--depth=1"] Nothing + let runFetch = callProcessInheritStderrStdout + (Cmd (Just acfDir) "git" menv ["fetch","--tags","--depth=1"]) + runFetch `C.catch` \(ex :: ReadProcessException) -> do + -- we failed, so wipe the directory and try again, see #1418 + $logWarn (T.pack (show ex)) + $logStickyDone "Failed to fetch package index, retrying." + removeDirRecur acfDir + readInNull suDir "git" menv cloneArgs Nothing + $logSticky "Fetching package index ..." + runFetch $logStickyDone "Fetched package index." - ignoringAbsence (removeFile tarFile) + when (indexGpgVerify index) - (readInNull acfDir - "git" - menv - ["tag","-v","current-hackage"] - (Just (T.unlines ["Signature verification failed. " - ,"Please ensure you've set up your" - ,"GPG keychain to accept the D6CF60FD signing key." - ,"For more information, see:" - ,"https://github.com/fpco/stackage-update#readme"]))) - $logDebug ("Exporting a tarball to " <> - (T.pack . toFilePath) tarFile) - deleteCache indexName' - let tarFileTmp = toFilePath tarFile ++ ".tmp" - readInNull acfDir - "git" - menv - ["archive" - ,"--format=tar" - ,"-o" - ,tarFileTmp - ,"current-hackage"] - Nothing - tarFileTmpPath <- parseAbsFile tarFileTmp - renameFile tarFileTmpPath tarFile + (readInNull acfDir "git" menv ["tag","-v","current-hackage"] + (Just (T.unlines ["Signature verification failed. " + ,"Please ensure you've set up your" + ,"GPG keychain to accept the D6CF60FD signing key." + ,"For more information, see:" + ,"https://github.com/fpco/stackage-update#readme"]))) + -- generate index archive when commit id differs from cloned repo + tarId <- getTarCommitId (toFilePath tarFile) + cloneId <- getCloneCommitId acfDir + unless (tarId `equals` cloneId) + (generateArchive acfDir tarFile) + where + getTarCommitId fp = + tryProcessStdout Nothing menv "sh" ["-c","git get-tar-commit-id < "++fp] + + getCloneCommitId dir = + tryProcessStdout (Just dir) menv "git" ["rev-parse","current-hackage^{}"] + + equals (Right cid1) (Right cid2) = cid1 == cid2 + equals _ _ = False + + generateArchive acfDir tarFile = do + ignoringAbsence (removeFile tarFile) + deleteCache indexName' + $logDebug ("Exporting a tarball to " <> (T.pack . toFilePath) tarFile) + let tarFileTmp = toFilePath tarFile ++ ".tmp" + readInNull acfDir + "git" menv ["archive","--format=tar","-o",tarFileTmp,"current-hackage"] + Nothing + tarFileTmpPath <- parseAbsFile tarFileTmp + renameFile tarFileTmpPath tarFile + -- | Update the index tarball via HTTP updateIndexHTTP :: (MonadIO m,MonadLogger m ,MonadThrow m,MonadReader env m,HasHttpManager env,HasConfig env) @@ -332,17 +353,77 @@ Left e -> $logDebug $ "Could not delete cache: " <> T.pack (show e) Right () -> $logDebug $ "Deleted index cache at " <> T.pack (toFilePath fp) --- | Load the cached package URLs, or created the cache if necessary. -getPackageCaches :: (MonadIO m, MonadLogger m, MonadReader env m, HasConfig env, MonadThrow m, HasHttpManager env, MonadBaseControl IO m, MonadCatch m) - => EnvOverride - -> m (Map PackageIdentifier (PackageIndex, PackageCache)) -getPackageCaches menv = do +-- | Lookup a package's versions from 'IO'. +getPackageVersionsIO + :: (MonadIO m, MonadReader env m, HasConfig env, MonadLogger m, HasHttpManager env, MonadBaseControl IO m, MonadCatch m) + => m (PackageName -> IO (Set Version)) +getPackageVersionsIO = do + getCaches <- getPackageCachesIO + return $ \name -> + fmap (lookupPackageVersions name) getCaches + +-- | Get the known versions for a given package from the package caches. +-- +-- See 'getPackageCaches' for performance notes. +getPackageVersions + :: (MonadIO m, MonadLogger m, MonadReader env m, HasConfig env, HasHttpManager env, MonadBaseControl IO m, MonadCatch m) + => PackageName + -> m (Set Version) +getPackageVersions pkgName = + fmap (lookupPackageVersions pkgName) getPackageCaches + +lookupPackageVersions :: PackageName -> Map PackageIdentifier a -> Set Version +lookupPackageVersions pkgName pkgCaches = + Set.fromList [v | PackageIdentifier n v <- Map.keys pkgCaches, n == pkgName] + +-- | Access the package caches from 'IO'. +-- +-- FIXME: This is a temporary solution until a better solution +-- to access the package caches from Stack.Build.ConstructPlan +-- has been found. +getPackageCachesIO + :: (MonadIO m, MonadReader env m, HasConfig env, MonadLogger m, HasHttpManager env, MonadBaseControl IO m, MonadCatch m) + => m (IO (Map PackageIdentifier (PackageIndex, PackageCache))) +getPackageCachesIO = toIO getPackageCaches + where + toIO :: (MonadIO m, MonadBaseControl IO m) => m a -> m (IO a) + toIO m = do + runInBase <- liftBaseWith $ \run -> return (void . run) + return $ do + i <- newIORef (error "Impossible evaluation in toIO") + runInBase $ do + x <- m + liftIO $ writeIORef i x + readIORef i + +-- | Load the package caches, or create the caches if necessary. +-- +-- This has two levels of caching: in memory, and the on-disk cache. So, +-- feel free to call this function multiple times. +getPackageCaches + :: (MonadIO m, MonadLogger m, MonadReader env m, HasConfig env, HasHttpManager env, MonadBaseControl IO m, MonadCatch m) + => m (Map PackageIdentifier (PackageIndex, PackageCache)) +getPackageCaches = do + menv <- getMinimalEnvOverride config <- askConfig - liftM mconcat $ forM (configPackageIndices config) $ \index -> do - fp <- configPackageIndexCache (indexName index) - PackageCacheMap pis' <- taggedDecodeOrLoad fp $ liftM PackageCacheMap $ populateCache menv index + mcached <- liftIO $ readIORef (configPackageCaches config) + case mcached of + Just cached -> return cached + Nothing -> do + result <- liftM mconcat $ forM (configPackageIndices config) $ \index -> do + fp <- configPackageIndexCache (indexName index) + PackageCacheMap pis' <- taggedDecodeOrLoad fp $ liftM PackageCacheMap $ populateCache menv index + return (fmap (index,) pis') + liftIO $ writeIORef (configPackageCaches config) (Just result) + return result - return (fmap (index,) pis') +-- | Clear the in-memory hackage index cache. This is needed when the +-- hackage index is updated. +clearPackageCaches :: (MonadIO m, MonadLogger m, MonadReader env m, HasConfig env, MonadThrow m, HasHttpManager env, MonadBaseControl IO m, MonadCatch m) + => m () +clearPackageCaches = do + cacheRef <- asks (configPackageCaches . getConfig) + liftIO $ writeIORef cacheRef Nothing --------------- Lifted from cabal-install, Distribution.Client.Tar: -- | Return the number of blocks in an entry. diff --git a/src/Stack/SDist.hs b/src/Stack/SDist.hs --- a/src/Stack/SDist.hs +++ b/src/Stack/SDist.hs @@ -122,7 +122,7 @@ getCabalLbs pvpBounds fp = do bs <- liftIO $ S.readFile fp (_warnings, gpd) <- readPackageUnresolvedBS Nothing bs - (_, _, _, _, sourceMap) <- loadSourceMap AllowNoTargets defaultBuildOpts + (_, _, _, _, sourceMap) <- loadSourceMap AllowNoTargets defaultBuildOptsCLI menv <- getMinimalEnvOverride (installedMap, _, _, _) <- getInstalled menv GetInstalledOpts { getInstalledProfiling = False @@ -176,7 +176,7 @@ readLocalPackage pkgDir = do cabalfp <- findOrGenerateCabalFile pkgDir name <- parsePackageNameFromFilePath cabalfp - config <- getPackageConfig defaultBuildOpts name + config <- getPackageConfig defaultBuildOptsCLI name (warnings,package) <- readPackage config cabalfp mapM_ (printCabalFileWarning cabalfp) warnings return LocalPackage @@ -189,7 +189,8 @@ , lpTestDeps = Map.empty , lpBenchDeps = Map.empty , lpTestBench = Nothing - , lpDirtyFiles = Just Set.empty + , lpForceDirty = False + , lpDirtyFiles = Nothing , lpNewBuildCache = Map.empty , lpFiles = Set.empty , lpComponents = Set.empty @@ -202,10 +203,11 @@ withSystemTempDir (stackProgName <> "-sdist") $ \tmpdir -> do menv <- getMinimalEnvOverride let bopts = defaultBuildOpts - baseConfigOpts <- mkBaseConfigOpts bopts - (_, _mbp, locals, _extraToBuild, _sourceMap) <- loadSourceMap NeedTargets bopts + let boptsCli = defaultBuildOptsCLI + baseConfigOpts <- mkBaseConfigOpts boptsCli + (_, _mbp, locals, _extraToBuild, _sourceMap) <- loadSourceMap NeedTargets boptsCli runInBase <- liftBaseWith $ \run -> return (void . run) - withExecuteEnv menv bopts baseConfigOpts locals + withExecuteEnv menv bopts boptsCli baseConfigOpts locals [] [] [] -- provide empty list of globals. This is a hack around custom Setup.hs files $ \ee -> withSingleContext runInBase ac ee task Nothing (Just "sdist") $ \_package cabalfp _pkgDir cabal _announce _console _mlogFile -> do @@ -260,7 +262,7 @@ -- and will throw an exception in case of critical errors. -- -- Note that we temporarily decompress the archive to analyze it. -checkSDistTarball :: (MonadIO m, MonadMask m, MonadThrow m, MonadCatch m, MonadLogger m, MonadReader env m, HasEnvConfig env) +checkSDistTarball :: (MonadIO m, MonadMask m, MonadThrow m, MonadLogger m, MonadReader env m, HasEnvConfig env) => Path Abs File -- ^ Absolute path to tarball -> m () checkSDistTarball tarball = withTempTarGzContents tarball $ \pkgDir' -> do @@ -269,7 +271,7 @@ -- ^ drop ".tar" ^ drop ".gz" cabalfp <- findOrGenerateCabalFile pkgDir name <- parsePackageNameFromFilePath cabalfp - config <- getPackageConfig defaultBuildOpts name + config <- getPackageConfig defaultBuildOptsCLI name (gdesc, pkgDesc) <- readPackageDescriptionDir config pkgDir $logInfo $ "Checking package '" <> packageNameText name <> "' for common mistakes" @@ -290,7 +292,7 @@ -- | Version of 'checkSDistTarball' that first saves lazy bytestring to -- temporary directory and then calls 'checkSDistTarball' on it. -checkSDistTarball' :: (MonadIO m, MonadMask m, MonadThrow m, MonadCatch m, MonadLogger m, MonadReader env m, HasEnvConfig env) +checkSDistTarball' :: (MonadIO m, MonadMask m, MonadThrow m, MonadLogger m, MonadReader env m, HasEnvConfig env) => String -- ^ Tarball name -> L.ByteString -- ^ Tarball contents as a byte string -> m () diff --git a/src/Stack/Setup.hs b/src/Stack/Setup.hs --- a/src/Stack/Setup.hs +++ b/src/Stack/Setup.hs @@ -38,7 +38,6 @@ import qualified Data.ByteString.Lazy as LBS import Data.Char (isSpace) import Data.Conduit (Conduit, ($$), (=$), await, yield, awaitForever) -import qualified Data.Conduit.Binary as CB import Data.Conduit.Lift (evalStateC) import qualified Data.Conduit.List as CL import Data.Either @@ -63,7 +62,7 @@ import Distribution.System (OS, Arch (..), Platform (..)) import qualified Distribution.System as Cabal import Distribution.Text (simpleParse) -import Language.Haskell.TH as TH +import Lens.Micro (set) import Network.HTTP.Client.Conduit import Network.HTTP.Download.Verified import Path @@ -80,7 +79,7 @@ import Stack.GhcPkg (createDatabase, getCabalPkgVer, getGlobalDB, mkGhcPackagePath) import Stack.Setup.Installed import Stack.Types -import Stack.Types.Internal (HasTerminal, HasReExec, HasLogLevel) +import Stack.Types.Internal (HasTerminal, HasReExec, HasLogLevel, envConfigBuildOpts, buildOptsInstallExes) import Stack.Types.StackT import qualified System.Directory as D import System.Environment (getExecutablePath) @@ -219,6 +218,7 @@ menv <- mkEnvOverride platform env compilerVer <- getCompilerVersion menv wc cabalVer <- getCabalPkgVer menv wc + $logDebug "Resolving package entries" packages <- mapM (resolvePackageEntry menv (bcRoot bconfig)) (bcPackageEntries bconfig) @@ -272,6 +272,13 @@ then Map.union utf8EnvVars else id) + $ case (soptsSkipMsys sopts, platform) of + (False, Platform Cabal.I386 Cabal.Windows) + -> Map.insert "MSYSTEM" "MINGW32" + (False, Platform Cabal.X86_64 Cabal.Windows) + -> Map.insert "MSYSTEM" "MINGW64" + _ -> id + -- For reasoning and duplication, see: https://github.com/fpco/stack/issues/70 $ Map.insert "HASKELL_PACKAGE_SANDBOX" (T.pack $ toFilePathNoTrailingSep deps) $ Map.insert "HASKELL_PACKAGE_SANDBOXES" @@ -286,6 +293,7 @@ , "" ]) $ Map.insert "HASKELL_DIST_DIR" (T.pack $ toFilePathNoTrailingSep distDir) env + () <- atomicModifyIORef envRef $ \m' -> (Map.insert es eo m', ()) return eo @@ -329,7 +337,9 @@ msystem <- if soptsUseSystem sopts - then getSystemCompiler menv0 wc + then do + $logDebug "Getting system compiler version" + getSystemCompiler menv0 wc else return Nothing Platform expectedArch _ <- asks getPlatform @@ -847,6 +857,7 @@ $logDebug $ "ziptool: " <> T.pack zipTool $logDebug $ "tar: " <> T.pack tarTool return $ do + ignoringAbsence (removeDirRecur destDir) ignoringAbsence (removeDirRecur unpackDir) readInNull destDir tarTool menv ["xf", toFilePath archiveFile] Nothing innerDir <- expectSingleUnpackedDir archiveFile destDir @@ -860,19 +871,19 @@ let stackYaml = unpackDir $(mkRelFile "stack.yaml") destBinDir = destDir $(mkRelDir "bin") ensureDir destBinDir - envConfig <- loadGhcjsEnvConfig stackYaml destBinDir + envConfig' <- loadGhcjsEnvConfig stackYaml destBinDir -- On windows we need to copy options files out of the install dir. Argh! -- This is done before the build, so that if it fails, things fail -- earlier. mwindowsInstallDir <- case platform of Platform _ Cabal.Windows -> - liftM Just $ runInnerStackT envConfig installationRootLocal + liftM Just $ runInnerStackT envConfig' installationRootLocal _ -> return Nothing $logSticky "Installing GHCJS (this will take a long time) ..." - runInnerStackT envConfig $ - build (\_ -> return ()) Nothing defaultBuildOpts { boptsInstallExes = True } + runInnerStackT ((set (envConfigBuildOpts.buildOptsInstallExes) True envConfig')) $ + (build (\_ -> return ()) Nothing defaultBuildOptsCLI) -- Copy over *.options files needed on windows. forM_ mwindowsInstallDir $ \dir -> do (_, files) <- listDir (dir $(mkRelDir "bin")) @@ -925,57 +936,77 @@ -- This only affects the case where GHCJS has been -- installed with an older version and not yet booted. stackYamlExists <- doesFileExist stackYaml - actualStackYaml <- if stackYamlExists then return stackYaml - else case cv of - GhcjsVersion version _ -> - liftM ((destDir $(mkRelDir "src")) ) $ - parseRelFile $ "ghcjs-" ++ versionString version ++ "/stack.yaml" + ghcjsVersion <- case cv of + GhcjsVersion version _ -> return version _ -> fail "ensureGhcjsBooted invoked on non GhcjsVersion" + actualStackYaml <- if stackYamlExists then return stackYaml + else + liftM ((destDir $(mkRelDir "src")) ) $ + parseRelFile $ "ghcjs-" ++ versionString ghcjsVersion ++ "/stack.yaml" actualStackYamlExists <- doesFileExist actualStackYaml unless actualStackYamlExists $ fail "Couldn't find GHCJS stack.yaml in old or new location." - bootGhcjs actualStackYaml destDir + bootGhcjs ghcjsVersion actualStackYaml destDir Left err -> throwM err bootGhcjs :: (MonadIO m, MonadBaseControl IO m, MonadLogger m, MonadCatch m, HasHttpManager env, HasTerminal env, HasReExec env, HasLogLevel env, MonadReader env m) - => Path Abs File -> Path Abs Dir -> m () -bootGhcjs stackYaml destDir = do + => Version -> Path Abs File -> Path Abs Dir -> m () +bootGhcjs ghcjsVersion stackYaml destDir = do envConfig <- loadGhcjsEnvConfig stackYaml (destDir $(mkRelDir "bin")) menv <- liftIO $ configEnvOverride (getConfig envConfig) defaultEnvSettings -- Install cabal-install if missing, or if the installed one is old. mcabal <- getCabalInstallVersion menv shouldInstallCabal <- case mcabal of Nothing -> do - $logInfo "No cabal-install binary found for use with GHCJS. Installing a local copy of cabal-install from source." + $logInfo "No cabal-install binary found for use with GHCJS." return True Just v | v < $(mkVersion "1.22.4") -> do $logInfo $ - "cabal-install found on PATH is too old to be used for booting GHCJS (version " <> + "The cabal-install found on PATH is too old to be used for booting GHCJS (version " <> versionText v <> - "). Installing a local copy of cabal-install from source." + ")." return True + | v >= $(mkVersion "1.23") -> do + $logWarn $ + "The cabal-install found on PATH is a version stack doesn't know about, version " <> + versionText v <> + ". This may or may not work.\n" <> + "See this issue: https://github.com/ghcjs/ghcjs/issues/470" + return False + | ghcjsVersion >= $(mkVersion "0.2.0.20160413") && v >= $(mkVersion "1.22.8") -> do + $logWarn $ + "The cabal-install found on PATH, version " <> + versionText v <> + ", is >= 1.22.8.\n" <> + "That version has a bug preventing ghcjs < 0.2.0.20160413 from booting.\n" <> + "See this issue: https://github.com/ghcjs/ghcjs/issues/470" + return True | otherwise -> return False + let envSettings = defaultEnvSettings { esIncludeGhcPackagePath = False } + menv' <- liftIO $ configEnvOverride (getConfig envConfig) envSettings when shouldInstallCabal $ do - $logSticky "Building cabal-install for use by ghcjs-boot ... " + $logInfo "Building a local copy of cabal-install from source." runInnerStackT envConfig $ build (\_ -> return ()) Nothing - defaultBuildOpts { boptsTargets = ["cabal-install"] } + defaultBuildOptsCLI { boptsCLITargets = ["cabal-install"] } + mcabal' <- getCabalInstallVersion menv' + case mcabal' of + Nothing -> + $logError $ + "Failed to get cabal-install version after installing it.\n" <> + "This shouldn't happen, because it gets built to the snapshot bin directory, which should be treated as being on the PATH." + Just v | v >= $(mkVersion "1.22.8") && v < $(mkVersion "1.23") -> + $logWarn $ + "Installed version of cabal-install is in a version range which may not work.\n" <> + "See this issue: https://github.com/ghcjs/ghcjs/issues/470\n" <> + "This version is specified by the stack.yaml file included in the ghcjs tarball.\n" + _ -> return () $logSticky "Booting GHCJS (this will take a long time) ..." - let envSettings = defaultEnvSettings { esIncludeGhcPackagePath = False } - menv' <- liftIO $ configEnvOverride (getConfig envConfig) envSettings - runAndLog Nothing "ghcjs-boot" menv' ["--clean"] + logProcessStderrStdout Nothing "ghcjs-boot" menv' ["--clean"] $logStickyDone "GHCJS booted." --- TODO: something similar is done in Stack.Build.Execute. Create some utilities --- for this? -runAndLog :: (MonadIO m, MonadBaseControl IO m, MonadLogger m) - => Maybe (Path Abs Dir) -> String -> EnvOverride -> [String] -> m () -runAndLog mdir name menv args = liftBaseWith $ \restore -> do - let logLines = CB.lines =$ CL.mapM_ (void . restore . monadLoggerLog $(TH.location >>= liftLoc) "" LevelInfo . toLogStr) - void $ restore $ sinkProcessStderrStdout mdir menv name args logLines logLines - loadGhcjsEnvConfig :: (MonadIO m, HasHttpManager r, MonadReader r m, HasTerminal r, HasReExec r, HasLogLevel r) => Path Abs File -> Path b t -> m EnvConfig loadGhcjsEnvConfig stackYaml binPath = runInnerStackLoggingT $ do @@ -1061,16 +1092,17 @@ msys <- parseRelDir $ "msys" ++ T.unpack (fromMaybe "32" $ T.stripPrefix "windows" osKey) withUnpackedTarball7z "MSYS2" si archiveFile archiveType (Just msys) destDir + + -- I couldn't find this officially documented anywhere, but you need to run + -- the MSYS shell once in order to initialize some pacman stuff. Once that + -- run happens, you can just run commands as usual. platform <- asks getPlatform menv0 <- getMinimalEnvOverride + newEnv0 <- modifyEnvOverride menv0 $ Map.insert "MSYSTEM" "MSYS" newEnv <- augmentPathMap [toFilePath $ destDir $(mkRelDir "usr") $(mkRelDir "bin")] - (unEnvOverride menv0) + (unEnvOverride newEnv0) menv <- mkEnvOverride platform newEnv - - -- I couldn't find this officially documented anywhere, but you need to run - -- the shell once in order to initialize some pacman stuff. Once that run - -- happens, you can just run commands as usual. runCmd (Cmd (Just destDir) "sh" menv ["--login", "-c", "true"]) Nothing -- No longer installing git, it's unreliable diff --git a/src/Stack/Setup/Installed.hs b/src/Stack/Setup/Installed.hs --- a/src/Stack/Setup/Installed.hs +++ b/src/Stack/Setup/Installed.hs @@ -88,10 +88,12 @@ getCompilerVersion menv wc = case wc of Ghc -> do + $logDebug "Asking GHC for its version" bs <- readProcessStdout Nothing menv "ghc" ["--numeric-version"] let (_, ghcVersion) = versionFromEnd bs GhcVersion <$> parseVersion (T.decodeUtf8 ghcVersion) Ghcjs -> do + $logDebug "Asking GHCJS for its version" -- Output looks like -- -- The Glorious Glasgow Haskell Compilation System for JavaScript, version 0.1.0 (GHC 7.10.2) @@ -117,17 +119,30 @@ , dir $(mkRelDir "mingw") $(mkRelDir "bin") ] } - (Platform _ Cabal.Windows, "msys2") -> return mempty + (Platform Cabal.I386 Cabal.Windows, "msys2") -> return mempty { edBins = goList - [ dir $(mkRelDir "usr") $(mkRelDir "bin") + [ dir $(mkRelDir "mingw32") $(mkRelDir "bin") + , dir $(mkRelDir "usr") $(mkRelDir "bin") + , dir $(mkRelDir "usr") $(mkRelDir "local") $(mkRelDir "bin") ] , edInclude = goList + [ dir $(mkRelDir "mingw32") $(mkRelDir "include") + ] + , edLib = goList + [ dir $(mkRelDir "mingw32") $(mkRelDir "lib") + ] + } + (Platform Cabal.X86_64 Cabal.Windows, "msys2") -> return mempty + { edBins = goList + [ dir $(mkRelDir "mingw64") $(mkRelDir "bin") + , dir $(mkRelDir "usr") $(mkRelDir "bin") + , dir $(mkRelDir "usr") $(mkRelDir "local") $(mkRelDir "bin") + ] + , edInclude = goList [ dir $(mkRelDir "mingw64") $(mkRelDir "include") - , dir $(mkRelDir "mingw32") $(mkRelDir "include") ] , edLib = goList [ dir $(mkRelDir "mingw64") $(mkRelDir "lib") - , dir $(mkRelDir "mingw32") $(mkRelDir "lib") ] } (_, isGHC -> True) -> return mempty diff --git a/src/Stack/Sig.hs b/src/Stack/Sig.hs --- a/src/Stack/Sig.hs +++ b/src/Stack/Sig.hs @@ -1,7 +1,3 @@ -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE RecordWildCards #-} -{-# LANGUAGE TemplateHaskell #-} - {-| Module : Stack.Sig Description : GPG Signatures for Stack @@ -12,54 +8,7 @@ Portability : POSIX -} -module Stack.Sig - ( module Sig - , sigCmdName - , sigSignCmdName - , sigSignHackageCmdName - , sigSignHackageOpts - , sigSignSdistCmdName - , sigSignSdistOpts - ) - where +module Stack.Sig (module Sig) where -import Options.Applicative import Stack.Sig.GPG as Sig import Stack.Sig.Sign as Sig - --- | The command name for dealing with signatures. -sigCmdName :: String -sigCmdName = "sig" - --- | The command name for signing packages. -sigSignCmdName :: String -sigSignCmdName = "sign" - --- | The command name for signing an sdist package file. -sigSignSdistCmdName :: String -sigSignSdistCmdName = "sdist" - --- | The command name for signing all your packages from hackage.org. -sigSignHackageCmdName :: String -sigSignHackageCmdName = "hackage" - --- | The URL of the running signature service to use (sig-service) -url :: Parser String -url = strOption - (long "url" <> - short 'u' <> - metavar "URL" <> - showDefault <> - value "https://sig.commercialhaskell.org") - --- | Signature sign (sdist) options -sigSignSdistOpts :: Parser (String, String) -sigSignSdistOpts = helper <*> - ((,) <$> url <*> - argument str (metavar "PATH")) - --- | Signature sign (hackage) options -sigSignHackageOpts :: Parser (String, String) -sigSignHackageOpts = helper <*> - ((,) <$> url <*> - argument str (metavar "USER")) diff --git a/src/Stack/Sig/GPG.hs b/src/Stack/Sig/GPG.hs --- a/src/Stack/Sig/GPG.hs +++ b/src/Stack/Sig/GPG.hs @@ -1,6 +1,7 @@ -{-# LANGUAGE CPP #-} -{-# LANGUAGE RankNTypes #-} -{-# LANGUAGE RecordWildCards #-} +{-# LANGUAGE CPP #-} +{-# LANGUAGE RankNTypes #-} +{-# LANGUAGE TemplateHaskell #-} +{-# LANGUAGE TupleSections #-} {-| Module : Stack.Sig.GPG @@ -12,91 +13,86 @@ Portability : POSIX -} -module Stack.Sig.GPG (fullFingerprint, signPackage, verifyFile) - where +module Stack.Sig.GPG (gpgSign, gpgVerify) where #if __GLASGOW_HASKELL__ < 710 -import Control.Applicative ((<$>)) +import Control.Applicative ((<$>), (<*>)) #endif import Control.Monad.Catch (MonadThrow, throwM) import Control.Monad.IO.Class (MonadIO, liftIO) import qualified Data.ByteString.Char8 as C -import Data.Char (isSpace) import Data.List (find) import Data.Monoid ((<>)) import qualified Data.Text as T import Path import Stack.Types +import System.Directory (findExecutable) import System.Exit (ExitCode(..)) -import System.Process (readProcessWithExitCode) - --- | Extract the full long @fingerprint@ given a short (or long) --- @fingerprint@ -fullFingerprint - :: (Monad m, MonadIO m, MonadThrow m) - => Fingerprint -> m Fingerprint -fullFingerprint (Fingerprint fp) = do - (code,out,err) <- - liftIO - (readProcessWithExitCode "gpg" ["--fingerprint", T.unpack fp] []) - if code /= ExitSuccess - then throwM (GPGFingerprintException (out ++ "\n" ++ err)) - else maybe - (throwM - (GPGFingerprintException - ("unable to extract full fingerprint from output:\n " <> - out))) - return - (let hasFingerprint = - (==) ["Key", "fingerprint", "="] . take 3 - fingerprint = - T.filter (not . isSpace) . T.pack . unwords . drop 3 - in Fingerprint . fingerprint <$> - find hasFingerprint (map words (lines out))) +import System.Process (ProcessHandle, runInteractiveProcess, + waitForProcess) +import System.IO (Handle, hGetContents, hPutStrLn) -- | Sign a file path with GPG, returning the @Signature@. -signPackage - :: (Monad m, MonadIO m, MonadThrow m) +gpgSign + :: (MonadIO m, MonadThrow m) => Path Abs File -> m Signature -signPackage path = do - (code,out,err) <- +gpgSign path = do + (_hIn,hOut,hErr,process) <- + gpg + [ "--output" + , "-" + , "--use-agent" + , "--detach-sig" + , "--armor" + , toFilePath path] + (out,err,code) <- liftIO - (readProcessWithExitCode - "gpg" - [ "--output" - , "-" - , "--use-agent" - , "--detach-sig" - , "--armor" - , toFilePath path] - []) + ((,,) <$> + hGetContents hOut <*> + hGetContents hErr <*> + waitForProcess process) if code /= ExitSuccess - then throwM (GPGSignException (out ++ "\n" ++ err)) - else return (Signature (C.pack out)) + then throwM (GPGSignException $ out <> "\n" <> err) + else return (Signature $ C.pack out) -- | Verify the @Signature@ of a file path returning the -- @Fingerprint@. -verifyFile - :: (Monad m, MonadIO m, MonadThrow m) +gpgVerify + :: (MonadIO m, MonadThrow m) => Signature -> Path Abs File -> m Fingerprint -verifyFile (Signature signature) path = do - let process = - readProcessWithExitCode - "gpg" - ["--verify", "-", toFilePath path] - (C.unpack signature) - (code,out,err) <- liftIO process +gpgVerify (Signature signature) path = do + (hIn,hOut,hErr,process) <- gpg ["--verify", "-", toFilePath path] + (_in,out,err,code) <- + liftIO + ((,,,) <$> + hPutStrLn hIn (C.unpack signature) <*> + hGetContents hOut <*> + hGetContents hErr <*> + waitForProcess process) if code /= ExitSuccess then throwM (GPGVerifyException (out ++ "\n" ++ err)) else maybe (throwM (GPGFingerprintException - ("unable to extract short fingerprint from output\n: " <> + ("unable to extract fingerprint from output\n: " <> out))) return - (let hasFingerprint = - (==) ["gpg:", "Signature", "made"] . take 3 - fingerprint = T.pack . last - in Fingerprint . fingerprint <$> - find hasFingerprint (map words (lines err))) + (mkFingerprint . T.pack . concat . drop 3 <$> + find + ((==) ["Primary", "key", "fingerprint:"] . take 3) + (map words (lines err))) + +-- | Try to execute `gpg2` but fallback to `gpg` (as a backup) +gpg + :: (MonadIO m, MonadThrow m) + => [String] -> m (Handle, Handle, Handle, ProcessHandle) +gpg args = do + mGpg2Path <- liftIO (findExecutable "gpg2") + case mGpg2Path of + Just _ -> liftIO (runInteractiveProcess "gpg2" args Nothing Nothing) + Nothing -> do + mGpgPath <- liftIO (findExecutable "gpg") + case mGpgPath of + Just _ -> liftIO (runInteractiveProcess "gpg" args Nothing Nothing) + Nothing -> throwM GPGNotFoundException diff --git a/src/Stack/Sig/Sign.hs b/src/Stack/Sig/Sign.hs --- a/src/Stack/Sig/Sign.hs +++ b/src/Stack/Sig/Sign.hs @@ -1,3 +1,4 @@ +{-# LANGUAGE CPP #-} {-# LANGUAGE FlexibleContexts #-} {-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE RankNTypes #-} @@ -13,42 +14,46 @@ Portability : POSIX -} -module Stack.Sig.Sign (sign, signTarBytes) where +module Stack.Sig.Sign (sign, signPackage, signTarBytes) where +#if __GLASGOW_HASKELL__ < 710 +import Control.Applicative (Applicative(..)) +#endif + import qualified Codec.Archive.Tar as Tar import qualified Codec.Compression.GZip as GZip import Control.Monad (when) import Control.Monad.Catch import Control.Monad.IO.Class import Control.Monad.Logger -import Control.Monad.Trans.Control import qualified Data.ByteString.Lazy as BS import qualified Data.ByteString.Lazy as L import Data.Monoid ((<>)) import qualified Data.Text as T -import Data.UUID (toString) -import Data.UUID.V4 (nextRandom) -import Network.HTTP.Conduit - (Response(..), RequestBody(..), Request(..), httpLbs, newManager, - tlsManagerSettings) +import Network.HTTP.Conduit (Response(..), RequestBody(..), + Request(..), httpLbs) +import Network.HTTP.Client (Manager) import Network.HTTP.Download import Network.HTTP.Types (status200, methodPut) import Path import Path.IO import Stack.Package -import qualified Stack.Sig.GPG as GPG +import Stack.Sig.GPG import Stack.Types import qualified System.FilePath as FP -- | Sign a haskell package with the given url of the signature -- service and a path to a tarball. sign - :: (MonadCatch m, MonadBaseControl IO m, MonadIO m, MonadMask m, MonadLogger m, MonadThrow m, MonadReader env m, HasConfig env) - => Maybe (Path Abs Dir) -> String -> Path Abs File -> m () -sign Nothing _ _ = throwM SigNoProjectRootException -sign (Just projectRoot) url filePath = do - withStackWorkTempDir - projectRoot +#if __GLASGOW_HASKELL__ < 710 + :: (Applicative m, MonadIO m, MonadLogger m, MonadMask m, MonadThrow m) +#else + :: (MonadIO m, MonadLogger m, MonadMask m, MonadThrow m) +#endif + => Manager -> String -> Path Abs File -> m Signature +sign manager url filePath = + withSystemTempDir + "stack" (\tempDir -> do bytes <- liftIO @@ -60,9 +65,9 @@ Nothing -> throwM SigInvalidSDistTarBall Just cabalPath -> do pkg <- cabalFilePackageId (tempDir cabalPath) - signPackage url pkg filePath) + signPackage manager url pkg filePath) where - extractCabalFile tempDir (Tar.Next entry entries) = do + extractCabalFile tempDir (Tar.Next entry entries) = case Tar.entryContent entry of (Tar.NormalFile lbs _) -> case FP.splitFileName (Tar.entryPath entry) of @@ -85,54 +90,42 @@ -- function will write the bytes to the path in a temp dir and sign -- the tarball with GPG. signTarBytes - :: (MonadCatch m, MonadBaseControl IO m, MonadIO m, MonadMask m, MonadLogger m, MonadThrow m, MonadReader env m, HasConfig env) - => Maybe (Path Abs Dir) -> String -> Path Rel File -> L.ByteString -> m () -signTarBytes Nothing _ _ _ = throwM SigNoProjectRootException -signTarBytes (Just projectRoot) url tarPath bs = - withStackWorkTempDir - projectRoot +#if __GLASGOW_HASKELL__ < 710 + :: (Applicative m, MonadIO m, MonadLogger m, MonadMask m, MonadThrow m) +#else + :: (MonadIO m, MonadLogger m, MonadMask m, MonadThrow m) +#endif + => Manager -> String -> Path Rel File -> L.ByteString -> m Signature +signTarBytes manager url tarPath bs = + withSystemTempDir + "stack" (\tempDir -> do let tempTarBall = tempDir tarPath liftIO (L.writeFile (toFilePath tempTarBall) bs) - sign (Just projectRoot) url tempTarBall) + sign manager url tempTarBall) -- | Sign a haskell package given the url to the signature service, a -- @PackageIdentifier@ and a file path to the package on disk. signPackage - :: (MonadCatch m, MonadBaseControl IO m, MonadIO m, MonadMask m, MonadLogger m, MonadThrow m) - => String -> PackageIdentifier -> Path Abs File -> m () -signPackage url pkg filePath = do - $logInfo ("GPG signing " <> T.pack (toFilePath filePath)) - sig@(Signature signature) <- GPG.signPackage filePath - let (PackageIdentifier n v) = pkg - name = show n - version = show v - verify <- GPG.verifyFile sig filePath - fingerprint <- GPG.fullFingerprint verify - req <- - parseUrl - (url <> "/upload/signature/" <> name <> "/" <> version <> "/" <> - T.unpack (fingerprintSample fingerprint)) + :: (MonadIO m, MonadLogger m, MonadThrow m) + => Manager -> String -> PackageIdentifier -> Path Abs File -> m Signature +signPackage manager url pkg filePath = do + sig@(Signature signature) <- gpgSign filePath + let (PackageIdentifier name version) = pkg + fingerprint <- gpgVerify sig filePath + let fullUrl = + url <> "/upload/signature/" <> show name <> "/" <> show version <> + "/" <> + show fingerprint + req <- parseUrl fullUrl let put = req { method = methodPut , requestBody = RequestBodyBS signature } - mgr <- liftIO (newManager tlsManagerSettings) - res <- liftIO (httpLbs put mgr) + res <- liftIO (httpLbs put manager) when (responseStatus res /= status200) (throwM (GPGSignException "unable to sign & upload package")) - -withStackWorkTempDir - :: (MonadCatch m, MonadIO m, MonadMask m, MonadLogger m, MonadReader env m, HasConfig env) - => Path Abs Dir -> (Path Abs Dir -> m ()) -> m () -withStackWorkTempDir projectRoot f = do - uuid <- liftIO nextRandom - uuidPath <- parseRelDir (toString uuid) - workDir <- getWorkDir - let tempDir = projectRoot workDir $(mkRelDir "tmp") uuidPath - bracket - (ensureDir tempDir) - (const (removeDirRecur tempDir)) - (const (f tempDir)) + $logInfo ("Signature uploaded to " <> T.pack fullUrl) + return sig diff --git a/src/Stack/Solver.hs b/src/Stack/Solver.hs --- a/src/Stack/Solver.hs +++ b/src/Stack/Solver.hs @@ -2,6 +2,8 @@ {-# LANGUAGE MultiParamTypeClasses #-} {-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE TemplateHaskell #-} +{-# LANGUAGE ScopedTypeVariables #-} +{-# LANGUAGE TypeFamilies #-} module Stack.Solver ( checkResolverSpec , cabalPackagesCheck @@ -27,7 +29,7 @@ import Data.Function (on) import qualified Data.HashMap.Strict as HashMap import Data.List ( (\\), isSuffixOf, intercalate - , minimumBy) + , minimumBy, isPrefixOf) import Data.List.Extra (groupSortOn) import Data.Map (Map) import qualified Data.Map as Map @@ -50,9 +52,11 @@ import Path.Find (findFiles) import Path.IO hiding (findExecutable, findFiles) import Prelude +import Safe (headMay) import Stack.BuildPlan import Stack.Constants (stackDotYaml) import Stack.Package (printCabalFileWarning + , hpack , readPackageUnresolved) import Stack.Setup import Stack.Setup.Installed @@ -98,7 +102,6 @@ : "--enable-tests" : "--enable-benchmarks" : "--dry-run" - : "--only-dependencies" : "--reorder-goals" : "--max-backjumps=-1" : "--package-db=clear" @@ -115,16 +118,19 @@ where errCheck = T.isInfixOf "Could not resolve dependencies" + cabalBuildErrMsg e = + ">>>> Cabal errors begin\n" + <> e + <> "<<<< Cabal errors end\n" parseCabalErrors err = do - let errExit e = error $ "Could not parse cabal-install errors:\n" - ++ (T.unpack e) + let errExit e = error $ "Could not parse cabal-install errors:\n\n" + ++ cabalBuildErrMsg (T.unpack e) msg = LT.toStrict $ decodeUtf8With lenientDecode err if errCheck msg then do - $logInfo "Attempt failed." - $logInfo "\n>>>> Cabal errors begin" - $logInfo $ msg <> "<<<< Cabal errors end\n" + $logInfo "Attempt failed.\n" + $logInfo $ cabalBuildErrMsg msg let pkgs = parseConflictingPkgs msg mPkgNames = map (C.simpleParse . T.unpack) pkgs pkgNames = map (fromCabalPackageName . C.pkgName) @@ -161,35 +167,47 @@ (errs, pairs) = partitionEithers $ map parseLine ls if null errs then return $ Right (Map.fromList pairs) - else error $ "Could not parse cabal-install output: " ++ show errs + else error $ "The following lines from cabal-install output could \ + \not be parsed: \n" + ++ (T.unpack (T.intercalate "\n" errs)) + -- An ugly parser to extract module id and flags + parseLine :: Text -> Either Text (PackageName, (Version, Map FlagName Bool)) parseLine t0 = maybe (Left t0) Right $ do - -- Sample output to parse: + -- Sample outputs to parse: -- text-1.2.1.1 (latest: 1.2.2.0) -integer-simple (via: parsec-3.1.9) (new package)) - -- An ugly parser to extract module id and flags - let t1 = T.concat $ - [ T.takeWhile (/= '(') - , (T.takeWhile (/= '(')) - . (T.drop 1) - . (T.dropWhile (/= ')')) - ] <*> [t0] + -- hspec-snap-1.0.0.0 *test (via: servant-snap-0.5) (new package) + -- time-locale-compat-0.1.1.1 -old-locale (via: http-api-data-0.2.2) (new package)) - ident':flags' <- Just $ T.words t1 - PackageIdentifier name version <- - parsePackageIdentifierFromString $ T.unpack ident' - flags <- mapM parseFlag flags' - Just (name, (version, Map.fromList flags)) + if (not $ T.null t0) then do + ident':rest <- Just $ T.words t0 + PackageIdentifier name version <- + parsePackageIdentifierFromString $ T.unpack ident' + + nextWord <- headMay rest + rest' <- case T.head nextWord of + '(' -> Just $ dropWhile (")" `T.isSuffixOf`) rest + '*' -> Just $ dropWhile ("*" `T.isPrefixOf`) rest + '+' -> Just rest + '-' -> Just rest + _ -> Nothing + + let fl = takeWhile (not . ("(" `T.isPrefixOf`)) rest' + flags <- mapM parseFlag fl + Just (name, (version, Map.fromList flags)) + else Nothing + + parseFlag :: Text -> Maybe (FlagName, Bool) parseFlag t0 = do - flag <- parseFlagNameFromString $ T.unpack t1 - return (flag, enabled) - where - (t1, enabled) = - case T.stripPrefix "-" t0 of - Nothing -> - case T.stripPrefix "+" t0 of - Nothing -> (t0, True) - Just x -> (x, True) - Just x -> (x, False) + (fl, st) <- case T.head t0 of + '-' -> Just $ (T.tail t0, False) + '+' -> Just $ (T.tail t0, True) + _ -> Nothing + if (not $ T.null fl) then do + flag <- parseFlagNameFromString $ T.unpack fl + return (flag, st) + else Nothing + toConstraintArgs userFlagMap = [formatFlagConstraint package flag enabled | (package, fs) <- Map.toList userFlagMap @@ -325,13 +343,6 @@ else error "Bug: An entry in flag map must have a corresponding \ \entry in the version map") -diffConstraints - :: (Eq v, Eq f) - => (v, f) -> (v, f) -> Maybe (v, f) -diffConstraints (v, f) (v', f') - | (v == v') && (f == f') = Nothing - | otherwise = Just (v, f) - -- | Given a resolver, user package constraints (versions and flags) and extra -- dependency constraints determine what extra dependencies are required -- outside the resolver snapshot and the specified extra dependencies. @@ -368,10 +379,10 @@ let -- Note - The order in Map.union below is important. -- We want to override snapshot with extra deps depConstraints = Map.union extraConstraints snapConstraints - -- Make sure deps do not include any src packages + -- Make sure to remove any user packages from the dep constraints -- There are two reasons for this: -- 1. We do not want snapshot versions to override the sources - -- 2. Sources may not have versions leading to bad cabal constraints + -- 2. Sources may have blank versions leading to bad cabal constraints depOnlyConstraints = Map.difference depConstraints srcConstraints solver t = cabalSolver menv cabalDirs t srcConstraints depOnlyConstraints $ @@ -411,6 +422,23 @@ snapConstraints) external = Map.union inSnapChanged extra + -- Just in case. + -- If cabal output contains versions of user packages, those + -- versions better be the same as those in our cabal file i.e. + -- cabal should not be solving using versions from external + -- indices. + let outVers = fmap fst srcs + inVers = fmap fst srcConstraints + bothVers = Map.intersectionWith (\v1 v2 -> (v1, v2)) + inVers outVers + when (not $ outVers `Map.isSubmapOf` inVers) $ do + let msg = "Error: user package versions returned by cabal \ + \solver are not the same as the versions in the \ + \cabal files:\n" + -- TODO We can do better in formatting the message + error $ T.unpack $ msg + <> (showItems $ map show (Map.toList bothVers)) + $logInfo $ "Successfully determined a build plan with " <> T.pack (show $ Map.size external) <> " external dependencies." @@ -419,7 +447,21 @@ Left x -> do $logInfo $ "*** Failed to arrive at a workable build plan." return $ Left x + where + -- Think of the first map as the deps reported in cabal output and + -- the second as the snapshot packages + -- Note: For flags we only require that the flags in cabal output be a + -- subset of the snapshot flags. This is to avoid a false difference + -- reporting due to any spurious flags in the build plan which will + -- always be absent in the cabal output. + diffConstraints + :: (Eq v, Eq a, Ord k) + => (v, Map k a) -> (v, Map k a) -> Maybe (v, Map k a) + diffConstraints (v, f) (v', f') + | (v == v') && (f `Map.isSubmapOf` f') = Nothing + | otherwise = Just (v, f) + -- | Given a resolver (snpashot, compiler or custom resolver) -- return the compiler version, package versions and packages flags -- for that resolver. @@ -432,18 +474,18 @@ -> Resolver -> m (CompilerVersion, Map PackageName (Version, Map FlagName Bool)) -getResolverConstraints stackYaml resolver - | ResolverSnapshot snapName <- resolver = do - mbp <- loadMiniBuildPlan snapName - return (mbpCompilerVersion mbp, mbpConstraints mbp) - | ResolverCustom _ url <- resolver = do - -- FIXME instead of passing the stackYaml dir we should maintain - -- the file URL in the custom resolver always relative to stackYaml. - mbp <- parseCustomMiniBuildPlan stackYaml url - return (mbpCompilerVersion mbp, mbpConstraints mbp) - | ResolverCompiler compiler <- resolver = - return (compiler, Map.empty) - | otherwise = error "Not reached" +getResolverConstraints stackYaml resolver = + case resolver of + ResolverSnapshot snapName -> do + mbp <- loadMiniBuildPlan snapName + return (mbpCompilerVersion mbp, mbpConstraints mbp) + ResolverCustom _ url -> do + -- FIXME instead of passing the stackYaml dir we should maintain + -- the file URL in the custom resolver always relative to stackYaml. + mbp <- parseCustomMiniBuildPlan stackYaml url + return (mbpCompilerVersion mbp, mbpConstraints mbp) + ResolverCompiler compiler -> + return (compiler, Map.empty) where mpiConstraints mpi = (mpiVersion mpi, mpiFlags mpi) mbpConstraints mbp = fmap mpiConstraints (mbpPackages mbp) @@ -469,23 +511,27 @@ -- TODO support custom resolver for stack init ResolverCustom {} -> return $ BuildPlanCheckPartial Map.empty Map.empty --- | Finds all files with a .cabal extension under a given directory. +-- | Finds all files with a .cabal extension under a given directory. If +-- a `hpack` `package.yaml` file exists, this will be used to generate a cabal +-- file. -- Subdirectories can be included depending on the @recurse@ parameter. findCabalFiles :: MonadIO m => Bool -> Path Abs Dir -> m [Path Abs File] -findCabalFiles recurse dir = - liftIO $ findFiles dir isCabal (\subdir -> recurse && not (isIgnored subdir)) +findCabalFiles recurse dir = liftIO $ do + findFiles dir isHpack subdirFilter >>= mapM_ (hpack . parent) + findFiles dir isCabal subdirFilter where - isCabal path = ".cabal" `isSuffixOf` toFilePath path + subdirFilter subdir = recurse && not (isIgnored subdir) + isHpack = (== "package.yaml") . toFilePath . filename + isCabal = (".cabal" `isSuffixOf`) . toFilePath - isIgnored path = FP.dropTrailingPathSeparator (toFilePath (dirname path)) - `Set.member` ignoredDirs + isIgnored path = "." `isPrefixOf` dirName || dirName `Set.member` ignoredDirs + where + dirName = FP.dropTrailingPathSeparator (toFilePath (dirname path)) -- | Special directories that we don't want to traverse for .cabal files ignoredDirs :: Set FilePath ignoredDirs = Set.fromList - [ ".git" - , "dist" - , ".stack-work" + [ "dist" ] -- | Perform some basic checks on a list of cabal files to be used for creating @@ -509,22 +555,30 @@ when (null cabalfps) $ error noPkgMsg - relpaths <- mapM makeRelativeToCurrentDir cabalfps + relpaths <- mapM prettyPath cabalfps $logInfo $ "Using cabal packages:" $logInfo $ T.pack (formatGroup relpaths) (warnings, gpds) <- fmap unzip (mapM readPackageUnresolved cabalfps) zipWithM_ (mapM_ . printCabalFileWarning) cabalfps warnings + -- package name cannot be empty or missing otherwise + -- it will result in cabal solver failure. + -- stack requires packages name to match the cabal file name + -- Just the latter check is enough to cover both the cases + let packages = zip cabalfps gpds - getEmptyNamePkg (fp, gpd) - | ((show . gpdPackageName) gpd) == "" = Just fp + getNameMismatchPkg (fp, gpd) + | (show . gpdPackageName) gpd /= (FP.takeBaseName . toFilePath) fp + = Just fp | otherwise = Nothing - emptyNamePkgs = mapMaybe getEmptyNamePkg packages + nameMismatchPkgs = mapMaybe getNameMismatchPkg packages - when (emptyNamePkgs /= []) $ do - rels <- mapM makeRelativeToCurrentDir emptyNamePkgs - error $ "Please assign a name to the following package(s):\n" + when (nameMismatchPkgs /= []) $ do + rels <- mapM prettyPath nameMismatchPkgs + error $ "Package name as defined in the .cabal file must match the \ + \.cabal file name.\n\ + \Please fix the following packages and try again:\n" <> (formatGroup rels) let dupGroups = filter ((> 1) . length) @@ -539,7 +593,7 @@ unique = packages \\ dupIgnored when (dupIgnored /= []) $ do - dups <- mapM (mapM (makeRelativeToCurrentDir . fst)) (dupGroups packages) + dups <- mapM (mapM (prettyPath. fst)) (dupGroups packages) $logWarn $ T.pack $ "Following packages have duplicate package names:\n" <> intercalate "\n" (map formatGroup dups) @@ -553,9 +607,8 @@ $ map (\(file, gpd) -> (gpdPackageName gpd,(file, gpd))) unique , map fst dupIgnored) -formatGroup :: [Path Rel File] -> String -formatGroup = concatMap formatPath - where formatPath path = "- " <> toFilePath path <> "\n" +formatGroup :: [String] -> String +formatGroup = concatMap (\path -> "- " <> path <> "\n") reportMissingCabalFiles :: (MonadIO m, MonadThrow m, MonadLogger m) => [Path Abs File] -- ^ Directories to scan @@ -564,7 +617,7 @@ reportMissingCabalFiles cabalfps includeSubdirs = do allCabalfps <- findCabalFiles includeSubdirs =<< getCurrentDir - relpaths <- mapM makeRelativeToCurrentDir (allCabalfps \\ cabalfps) + relpaths <- mapM prettyPath (allCabalfps \\ cabalfps) unless (null relpaths) $ do $logWarn $ "The following packages are missing from the config:" $logWarn $ T.pack (formatGroup relpaths) @@ -711,3 +764,12 @@ , " - Add any missing remote packages.\n" , " - Add extra dependencies to guide solver.\n" ] + +prettyPath + :: forall r t m. (MonadIO m, RelPath (Path r t) ~ Path Rel t, AnyPath (Path r t)) + => Path r t -> m String +prettyPath path = do + eres <- liftIO $ try $ makeRelativeToCurrentDir path + return $ case eres of + Left (_ :: PathParseException) -> toFilePath path + Right res -> toFilePath (res :: Path Rel t) diff --git a/src/Stack/Types.hs b/src/Stack/Types.hs --- a/src/Stack/Types.hs +++ b/src/Stack/Types.hs @@ -16,6 +16,7 @@ import Stack.Types.Nix as X import Stack.Types.Image as X import Stack.Types.Build as X +import Stack.Types.Urls as X import Stack.Types.Package as X import Stack.Types.Compiler as X import Stack.Types.Sig as X diff --git a/src/Stack/Types/Build.hs b/src/Stack/Types/Build.hs --- a/src/Stack/Types/Build.hs +++ b/src/Stack/Types/Build.hs @@ -34,6 +34,7 @@ ,ConfigCache(..) ,ConstructPlanException(..) ,configureOpts + ,isStackOpt ,BadDependency(..) ,wantedLocalPackages ,FileCacheInfo (..) @@ -66,14 +67,14 @@ import Distribution.System (Arch) import Distribution.PackageDescription (TestSuiteInterface) import Distribution.Text (display) -import GHC.Generics +import GHC.Generics (Generic, from, to) import Path (Path, Abs, File, Dir, mkRelDir, toFilePath, parseRelDir, ()) import Path.Extra (toFilePathNoTrailingSep) import Prelude -import Stack.Types.FlagName -import Stack.Types.GhcPkgId import Stack.Types.Compiler import Stack.Types.Config +import Stack.Types.FlagName +import Stack.Types.GhcPkgId import Stack.Types.Package import Stack.Types.PackageIdentifier import Stack.Types.PackageName @@ -123,6 +124,8 @@ | SolverGiveUp String | SolverMissingCabalInstall | SomeTargetsNotBuildable [(PackageName, NamedComponent)] + | TestSuiteExeMissing Bool String String String + | CabalCopyFailed Bool String deriving Typeable data FlagSource = FSCommandLine | FSStackYaml @@ -334,7 +337,36 @@ "The following components have 'buildable: False' set in the cabal configuration, and so cannot be targets:\n " ++ T.unpack (renderPkgComponents xs) ++ "\nTo resolve this, either provide flags such that these components are buildable, or only specify buildable targets." + show (TestSuiteExeMissing isSimpleBuildType exeName pkgName testName) = + missingExeError isSimpleBuildType $ concat + [ "Test suite executable \"" + , exeName + , " not found for " + , pkgName + , ":test:" + , testName + ] + show (CabalCopyFailed isSimpleBuildType innerMsg) = + missingExeError isSimpleBuildType $ concat + [ "'cabal copy' failed. Error message:\n" + , innerMsg + , "\n" + ] +missingExeError :: Bool -> String -> String +missingExeError isSimpleBuildType msg = + unlines $ msg : + case possibleCauses of + [] -> [] + [cause] -> ["One possible cause of this issue is:\n* " <> cause] + _ -> "Possible causes of this issue:" : map ("* " <>) possibleCauses + where + possibleCauses = + "No module named \"Main\". The 'main-is' source file should usually have a header indicating that it's a 'Main' module." : + if isSimpleBuildType + then [] + else ["The Setup.hs file is changing the installation target dir."] + instance Exception StackBuildException data ConstructPlanException @@ -413,119 +445,7 @@ ---------------------------------------------- --- | Which subset of packages to build -data BuildSubset - = BSAll - | BSOnlySnapshot - -- ^ Only install packages in the snapshot database, skipping - -- packages intended for the local database. - | BSOnlyDependencies - deriving (Show, Eq) --- | Configuration for building. -data BuildOpts = - BuildOpts {boptsTargets :: ![Text] - ,boptsLibProfile :: !Bool - ,boptsExeProfile :: !Bool - ,boptsHaddock :: !Bool - -- ^ Build haddocks? - ,boptsHaddockDeps :: !(Maybe Bool) - -- ^ Build haddocks for dependencies? - ,boptsDryrun :: !Bool - ,boptsGhcOptions :: ![Text] - ,boptsFlags :: !(Map (Maybe PackageName) (Map FlagName Bool)) - ,boptsInstallExes :: !Bool - -- ^ Install executables to user path after building? - ,boptsPreFetch :: !Bool - -- ^ Fetch all packages immediately - ,boptsBuildSubset :: !BuildSubset - ,boptsFileWatch :: !FileWatchOpts - -- ^ Watch files for changes and automatically rebuild - ,boptsKeepGoing :: !(Maybe Bool) - -- ^ Keep building/running after failure - ,boptsForceDirty :: !Bool - -- ^ Force treating all local packages as having dirty files - - ,boptsTests :: !Bool - -- ^ Turn on tests for local targets - ,boptsTestOpts :: !TestOpts - -- ^ Additional test arguments - - ,boptsBenchmarks :: !Bool - -- ^ Turn on benchmarks for local targets - ,boptsBenchmarkOpts :: !BenchmarkOpts - -- ^ Additional test arguments - ,boptsExec :: ![(String, [String])] - -- ^ Commands (with arguments) to run after a successful build - ,boptsOnlyConfigure :: !Bool - -- ^ Only perform the configure step when building - ,boptsReconfigure :: !Bool - -- ^ Perform the configure step even if already configured - ,boptsCabalVerbose :: !Bool - -- ^ Ask Cabal to be verbose in its builds - } - deriving (Show) - -defaultBuildOpts :: BuildOpts -defaultBuildOpts = BuildOpts - { boptsTargets = [] - , boptsLibProfile = False - , boptsExeProfile = False - , boptsHaddock = False - , boptsHaddockDeps = Nothing - , boptsDryrun = False - , boptsGhcOptions = [] - , boptsFlags = Map.empty - , boptsInstallExes = False - , boptsPreFetch = False - , boptsBuildSubset = BSAll - , boptsFileWatch = NoFileWatch - , boptsKeepGoing = Nothing - , boptsForceDirty = False - , boptsTests = False - , boptsTestOpts = defaultTestOpts - , boptsBenchmarks = False - , boptsBenchmarkOpts = defaultBenchmarkOpts - , boptsExec = [] - , boptsOnlyConfigure = False - , boptsReconfigure = False - , boptsCabalVerbose = False - } - --- | Options for the 'FinalAction' 'DoTests' -data TestOpts = - TestOpts {toRerunTests :: !Bool -- ^ Whether successful tests will be run gain - ,toAdditionalArgs :: ![String] -- ^ Arguments passed to the test program - ,toCoverage :: !Bool -- ^ Generate a code coverage report - ,toDisableRun :: !Bool -- ^ Disable running of tests - } deriving (Eq,Show) - -defaultTestOpts :: TestOpts -defaultTestOpts = TestOpts - { toRerunTests = True - , toAdditionalArgs = [] - , toCoverage = False - , toDisableRun = False - } - --- | Options for the 'FinalAction' 'DoBenchmarks' -data BenchmarkOpts = - BenchmarkOpts {beoAdditionalArgs :: !(Maybe String) -- ^ Arguments passed to the benchmark program - ,beoDisableRun :: !Bool -- ^ Disable running of benchmarks - } deriving (Eq,Show) - -defaultBenchmarkOpts :: BenchmarkOpts -defaultBenchmarkOpts = BenchmarkOpts - { beoAdditionalArgs = Nothing - , beoDisableRun = False - } - -data FileWatchOpts - = NoFileWatch - | FileWatch - | FileWatchPoll - deriving (Show,Eq) - -- | Package dependency oracle. newtype PkgDepsOracle = PkgDeps PackageName @@ -627,8 +547,10 @@ , bcoSnapInstallRoot :: !(Path Abs Dir) , bcoLocalInstallRoot :: !(Path Abs Dir) , bcoBuildOpts :: !BuildOpts + , bcoBuildOptsCLI :: !BuildOptsCLI , bcoExtraDBs :: ![(Path Abs Dir)] } + deriving Show -- | Render a @BaseConfigOpts@ to an actual list of options configureOpts :: EnvConfig @@ -644,6 +566,29 @@ , coNoDirs = configureOptsNoDir econfig bco deps wanted isLocal package } +-- options set by stack +isStackOpt :: Text -> Bool +isStackOpt t = any (`T.isPrefixOf` t) + [ "--dependency=" + , "--constraint=" + , "--package-db=" + , "--libdir=" + , "--bindir=" + , "--datadir=" + , "--libexecdir=" + , "--sysconfdir" + , "--docdir=" + , "--htmldir=" + , "--haddockdir=" + , "--enable-tests" + , "--enable-benchmarks" + , "--enable-library-profiling" + , "--enable-executable-profiling" + , "--exact-configuration" + ] || elem t + [ "--user" + ] + configureOptsDirs :: BaseConfigOpts -> InstallLocation -> Package @@ -687,31 +632,43 @@ configureOptsNoDir econfig bco deps wanted isLocal package = concat [ depOptions , ["--enable-library-profiling" | boptsLibProfile bopts || boptsExeProfile bopts] - , ["--enable-executable-profiling" | boptsExeProfile bopts] + , ["--enable-executable-profiling" | boptsExeProfile bopts && isLocal] + , ["--enable-split-objs" | boptsSplitObjs bopts] , map (\(name,enabled) -> "-f" <> (if enabled then "" else "-") <> flagNameString name) - (Map.toList (packageFlags package)) + (Map.toList flags) , concatMap (\x -> ["--ghc-options", T.unpack x]) allGhcOptions , map (("--extra-include-dirs=" ++) . T.unpack) (Set.toList (configExtraIncludeDirs config)) , map (("--extra-lib-dirs=" ++) . T.unpack) (Set.toList (configExtraLibDirs config)) , if whichCompiler (envConfigCompilerVersion econfig) == Ghcjs then ["--ghcjs"] else [] + , if useExactConf then ["--exact-configuration"] else [] ] where config = getConfig econfig bopts = bcoBuildOpts bco + boptsCli = bcoBuildOptsCLI bco + -- TODO: instead always enable this when the cabal version is new + -- enough. That way we'll detect bugs with --exact-configuration + -- earlier. Cabal also might do less work then. + useExactConf = configAllowNewer config + + newerCabal = envConfigCabalVersion econfig >= $(mkVersion "1.22") + + -- Unioning atop defaults is needed so that all flags are specified + -- with --exact-configuration. + flags | useExactConf = packageFlags package `Map.union` packageDefaultFlags package + | otherwise = packageFlags package + depOptions = map (uncurry toDepOption) $ Map.toList deps where - toDepOption = - if envConfigCabalVersion econfig >= $(mkVersion "1.22") - then toDepOption1_22 - else toDepOption1_18 + toDepOption = if newerCabal then toDepOption1_22 else toDepOption1_18 toDepOption1_22 ident gid = concat [ "--dependency=" @@ -733,8 +690,11 @@ [ Map.findWithDefault [] Nothing (configGhcOptions config) , Map.findWithDefault [] (Just $ packageName package) (configGhcOptions config) , concat [["-fhpc"] | isLocal && toCoverage (boptsTestOpts bopts)] + , if (boptsLibProfile bopts || boptsExeProfile bopts) + then ["-auto-all","-caf-all"] + else [] , if includeExtraOptions - then boptsGhcOptions bopts + then boptsCLIGhcOptions boptsCli else [] ] diff --git a/src/Stack/Types/Config.hs b/src/Stack/Types/Config.hs --- a/src/Stack/Types/Config.hs +++ b/src/Stack/Types/Config.hs @@ -43,6 +43,7 @@ ,EnvConfig(..) ,HasEnvConfig(..) ,getWhichCompiler + ,getCompilerPath -- * Details -- ** ApplyGhcOptions ,ApplyGhcOptions(..) @@ -61,7 +62,7 @@ ,LoadConfig(..) -- ** PackageEntry & PackageLocation ,PackageEntry(..) - ,peExtraDep + ,TreatLikeExtraDep ,PackageLocation(..) ,RemotePackageType(..) -- ** PackageIndex, IndexName & IndexLocation @@ -91,7 +92,7 @@ ,bindirSuffix ,configInstalledCache ,configMiniBuildPlanCache - ,configProjectWorkDir + ,getProjectWorkDir ,docDirSuffix ,flagCacheLocal ,extraBinDirs @@ -120,12 +121,13 @@ -- ** Docker entrypoint ,DockerEntrypoint(..) ,DockerUser(..) + ,module X ) where import Control.Applicative import Control.Arrow ((&&&)) import Control.Exception -import Control.Monad (liftM, mzero, forM) +import Control.Monad (liftM, mzero, forM, join) import Control.Monad.Catch (MonadThrow, throwM) import Control.Monad.Logger (LogLevel(..)) import Control.Monad.Reader (MonadReader, ask, asks, MonadIO, liftIO) @@ -139,7 +141,10 @@ import Data.ByteString (ByteString) import qualified Data.ByteString.Char8 as S8 import Data.Either (partitionEithers) +import Data.IORef (IORef) import Data.List (stripPrefix) +import Data.List.NonEmpty (NonEmpty) +import qualified Data.List.NonEmpty as NonEmpty import Data.Hashable (Hashable) import Data.Map (Map) import qualified Data.Map as Map @@ -159,8 +164,8 @@ import Network.HTTP.Client (parseUrl) import Path import qualified Paths_stack as Meta -import {-# SOURCE #-} Stack.Constants (stackRootEnvVar) import Stack.Types.BuildPlan (SnapName, renderSnapName, parseSnapName) +import Stack.Types.Urls import Stack.Types.Compiler import Stack.Types.Docker import Stack.Types.Nix @@ -172,7 +177,11 @@ import Stack.Types.TemplateName import Stack.Types.Version import System.PosixCompat.Types (UserID, GroupID, FileMode) -import System.Process.Read (EnvOverride) +import System.Process.Read (EnvOverride, findExecutable) + +-- Re-exports +import Stack.Types.Config.Build as X + #ifdef mingw32_HOST_OS import qualified Crypto.Hash.SHA1 as SHA1 import qualified Data.ByteString.Base16 as B16 @@ -186,6 +195,8 @@ -- ^ this allows to override .stack-work directory ,configUserConfigPath :: !(Path Abs File) -- ^ Path to user configuration file (usually ~/.stack/config.yaml) + ,configBuild :: !BuildOpts + -- ^ Build configuration ,configDocker :: !DockerOpts -- ^ Docker configuration ,configNix :: !NixOpts @@ -209,9 +220,12 @@ -- ^ The variant of GHC requested by the user. -- In most cases, use 'BuildConfig' or 'MiniConfig's version instead, -- which will have an auto-detected default. - ,configLatestSnapshotUrl :: !Text - -- ^ URL for a JSON file containing information on the latest - -- snapshots available. + ,configUrls :: !Urls + -- ^ URLs for other files used by stack. + -- TODO: Better document + -- e.g. The latest snapshot file. + -- A build plan name (e.g. lts5.9.yaml) is appended when downloading + -- the build plan actually. ,configPackageIndices :: ![PackageIndex] -- ^ Information on package indices. This is left biased, meaning that -- packages in an earlier index will shadow those in a later index. @@ -282,6 +296,9 @@ ,configAllowDifferentUser :: !Bool -- ^ Allow users other than the stack root owner to use the stack -- installation. + ,configPackageCaches :: !(IORef (Maybe (Map PackageIdentifier (PackageIndex, PackageCache)))) + -- ^ In memory cache of hackage index. + ,configMaybeProject :: !(Maybe (Project, Path Abs File)) } -- | Which packages to ghc-options on the command line apply to? @@ -455,8 +472,7 @@ , bcWantedCompiler :: !CompilerVersion -- ^ Compiler version wanted for this build , bcPackageEntries :: ![PackageEntry] - -- ^ Local packages identified by a path, Bool indicates whether it is - -- a non-dependency (the opposite of 'peExtraDep') + -- ^ Local packages , bcExtraDeps :: !(Map PackageName Version) -- ^ Extra dependencies specified in configuration. -- @@ -476,8 +492,6 @@ -- for providing better error messages. , bcGHCVariant :: !GHCVariant -- ^ The variant of GHC used to select a GHC bindist. - , bcPackageCaches :: !(Map PackageIdentifier (PackageIndex, PackageCache)) - -- ^ Shared package cache map } -- | Directory containing the project's stack.yaml file @@ -495,7 +509,7 @@ {envConfigBuildConfig :: !BuildConfig ,envConfigCabalVersion :: !Version ,envConfigCompilerVersion :: !CompilerVersion - ,envConfigPackages :: !(Map (Path Abs Dir) Bool)} + ,envConfigPackages :: !(Map (Path Abs Dir) TreatLikeExtraDep)} instance HasBuildConfig EnvConfig where getBuildConfig = envConfigBuildConfig instance HasConfig EnvConfig @@ -518,31 +532,21 @@ } data PackageEntry = PackageEntry - { peExtraDepMaybe :: !(Maybe Bool) - -- ^ Is this package a dependency? This means the local package will be - -- treated just like an extra-deps: it will only be built as a dependency - -- for others, and its test suite/benchmarks will not be run. - -- - -- Useful modifying an upstream package, see: - -- https://github.com/commercialhaskell/stack/issues/219 - -- https://github.com/commercialhaskell/stack/issues/386 - , peValidWanted :: !(Maybe Bool) - -- ^ Deprecated name meaning the opposite of peExtraDep. Only present to - -- provide deprecation warnings to users. + { peExtraDep :: !TreatLikeExtraDep , peLocation :: !PackageLocation , peSubdirs :: ![FilePath] } deriving Show --- | Once peValidWanted is removed, this should just become the field name in PackageEntry. -peExtraDep :: PackageEntry -> Bool -peExtraDep pe = - case peExtraDepMaybe pe of - Just x -> x - Nothing -> - case peValidWanted pe of - Just x -> not x - Nothing -> False +-- | Should a package be treated just like an extra-dep? +-- +-- 'True' means, it will only be built as a dependency +-- for others, and its test suite/benchmarks will not be run. +-- +-- Useful modifying an upstream package, see: +-- https://github.com/commercialhaskell/stack/issues/219 +-- https://github.com/commercialhaskell/stack/issues/386 +type TreatLikeExtraDep = Bool instance ToJSON PackageEntry where toJSON pe | not (peExtraDep pe) && null (peSubdirs pe) = @@ -556,15 +560,13 @@ parseJSON (String t) = do WithJSONWarnings loc _ <- parseJSON $ String t return $ noJSONWarnings - (PackageEntry - { peExtraDepMaybe = Nothing - , peValidWanted = Nothing + (PackageEntry + { peExtraDep = False , peLocation = loc , peSubdirs = [] }) parseJSON v = withObjectWarnings "PackageEntry" (\o -> PackageEntry - <$> o ..:? "extra-dep" - <*> o ..:? "valid-wanted" + <$> o ..:? "extra-dep" ..!= False <*> jsonSubWarnings (o ..: "location") <*> o ..:? "subdirs" ..!= []) v @@ -742,8 +744,12 @@ -- Configurations may be "cascaded" using mappend (left-biased). data ConfigMonoid = ConfigMonoid - { configMonoidWorkDir :: !(Maybe FilePath) + { configMonoidStackRoot :: !(Maybe (Path Abs Dir)) + -- ^ See: 'configStackRoot' + , configMonoidWorkDir :: !(Maybe FilePath) -- ^ See: 'configWorkDir'. + , configMonoidBuildOpts :: !BuildOptsMonoid + -- ^ build options. , configMonoidDockerOpts :: !DockerOptsMonoid -- ^ Docker options. , configMonoidNixOpts :: !NixOptsMonoid @@ -753,7 +759,9 @@ , configMonoidHideTHLoading :: !(Maybe Bool) -- ^ See: 'configHideTHLoading' , configMonoidLatestSnapshotUrl :: !(Maybe Text) - -- ^ See: 'configLatestSnapshotUrl' + -- ^ Deprecated in favour of 'urlsMonoidLatestSnapshot' + , configMonoidUrls :: !UrlsMonoid + -- ^ See: 'configUrls , configMonoidPackageIndices :: !(Maybe [PackageIndex]) -- ^ See: 'configPackageIndices' , configMonoidSystemGHC :: !(Maybe Bool) @@ -819,12 +827,15 @@ instance Monoid ConfigMonoid where mempty = ConfigMonoid - { configMonoidWorkDir = Nothing + { configMonoidStackRoot = Nothing + , configMonoidWorkDir = Nothing + , configMonoidBuildOpts = mempty , configMonoidDockerOpts = mempty , configMonoidNixOpts = mempty , configMonoidConnectionCount = Nothing , configMonoidHideTHLoading = Nothing , configMonoidLatestSnapshotUrl = Nothing + , configMonoidUrls = mempty , configMonoidPackageIndices = Nothing , configMonoidSystemGHC = Nothing , configMonoidInstallGHC = Nothing @@ -856,12 +867,15 @@ , configMonoidAllowDifferentUser = Nothing } mappend l r = ConfigMonoid - { configMonoidWorkDir = configMonoidWorkDir l <|> configMonoidWorkDir r + { configMonoidStackRoot = configMonoidStackRoot l <|> configMonoidStackRoot r + , configMonoidWorkDir = configMonoidWorkDir l <|> configMonoidWorkDir r + , configMonoidBuildOpts = configMonoidBuildOpts l <> configMonoidBuildOpts r , configMonoidDockerOpts = configMonoidDockerOpts l <> configMonoidDockerOpts r , configMonoidNixOpts = configMonoidNixOpts l <> configMonoidNixOpts r , configMonoidConnectionCount = configMonoidConnectionCount l <|> configMonoidConnectionCount r , configMonoidHideTHLoading = configMonoidHideTHLoading l <|> configMonoidHideTHLoading r , configMonoidLatestSnapshotUrl = configMonoidLatestSnapshotUrl l <|> configMonoidLatestSnapshotUrl r + , configMonoidUrls = configMonoidUrls l <> configMonoidUrls r , configMonoidPackageIndices = configMonoidPackageIndices l <|> configMonoidPackageIndices r , configMonoidSystemGHC = configMonoidSystemGHC l <|> configMonoidSystemGHC r , configMonoidInstallGHC = configMonoidInstallGHC l <|> configMonoidInstallGHC r @@ -902,12 +916,16 @@ -- warnings for missing fields. parseConfigMonoidJSON :: Object -> WarningParser ConfigMonoid parseConfigMonoidJSON obj = do + -- Parsing 'stackRoot' from 'stackRoot'/config.yaml would be nonsensical + let configMonoidStackRoot = Nothing configMonoidWorkDir <- obj ..:? configMonoidWorkDirName + configMonoidBuildOpts <- jsonSubWarnings (obj ..:? configMonoidBuildOptsName ..!= mempty) configMonoidDockerOpts <- jsonSubWarnings (obj ..:? configMonoidDockerOptsName ..!= mempty) configMonoidNixOpts <- jsonSubWarnings (obj ..:? configMonoidNixOptsName ..!= mempty) configMonoidConnectionCount <- obj ..:? configMonoidConnectionCountName configMonoidHideTHLoading <- obj ..:? configMonoidHideTHLoadingName configMonoidLatestSnapshotUrl <- obj ..:? configMonoidLatestSnapshotUrlName + configMonoidUrls <- jsonSubWarnings (obj ..:? configMonoidUrlsName ..!= mempty) configMonoidPackageIndices <- jsonSubWarningsTT (obj ..:? configMonoidPackageIndicesName) configMonoidSystemGHC <- obj ..:? configMonoidSystemGHCName configMonoidInstallGHC <- obj ..:? configMonoidInstallGHCName @@ -986,6 +1004,9 @@ configMonoidWorkDirName :: Text configMonoidWorkDirName = "work-dir" +configMonoidBuildOptsName :: Text +configMonoidBuildOptsName = "build" + configMonoidDockerOptsName :: Text configMonoidDockerOptsName = "docker" @@ -1001,6 +1022,9 @@ configMonoidLatestSnapshotUrlName :: Text configMonoidLatestSnapshotUrlName = "latest-snapshot-url" +configMonoidUrlsName :: Text +configMonoidUrlsName = "urls" + configMonoidPackageIndicesName :: Text configMonoidPackageIndicesName = "package-indices" @@ -1095,12 +1119,12 @@ | UnexpectedArchiveContents [Path Abs Dir] [Path Abs File] | UnableToExtractArchive Text (Path Abs File) | BadStackVersionException VersionRange - | NoMatchingSnapshot [SnapName] + | NoMatchingSnapshot (NonEmpty SnapName) | ResolverMismatch Resolver String | ResolverPartial Resolver String | NoSuchDirectory FilePath | ParseGHCVariantException String - | BadStackRootEnvVar (Path Abs Dir) + | BadStackRoot (Path Abs Dir) | Won'tCreateStackRootInDirectoryOwnedByDifferentUser (Path Abs Dir) (Path Abs Dir) -- ^ @$STACK_ROOT@, parent dir | UserDoesn'tOwnDirectory (Path Abs Dir) deriving Typeable @@ -1148,7 +1172,7 @@ [ "None of the following snapshots provides a compiler matching " , "your package(s):\n" , unlines $ map (\name -> " - " <> T.unpack (renderSnapName name)) - names + (NonEmpty.toList names) , "\nYou can try the following options:\n" , " - Use '--omit-packages to exclude mismatching package(s).\n" , " - Use '--resolver' to specify a matching snapshot/resolver\n" @@ -1168,6 +1192,7 @@ , "' does not have all the packages to match your requirements.\n" , unlines $ fmap (" " <>) (lines errDesc) , "\nHowever, you can try '--solver' to use external packages." + , "\nUse '--omit-packages' if you want to create a config anyway." ] show (NoSuchDirectory dir) = concat ["No directory could be located matching the supplied path: " @@ -1177,17 +1202,13 @@ [ "Invalid ghc-variant value: " , v ] - show (BadStackRootEnvVar envStackRoot) = concat - [ "Invalid $" - , stackRootEnvVar - , ": '" - , toFilePath envStackRoot + show (BadStackRoot stackRoot) = concat + [ "Invalid stack root: '" + , toFilePath stackRoot , "'. Please provide a valid absolute path." ] show (Won'tCreateStackRootInDirectoryOwnedByDifferentUser envStackRoot parentDir) = concat - [ "Preventing creation of $" - , stackRootEnvVar - , " '" + [ "Preventing creation of stack root '" , toFilePath envStackRoot , "'. Parent directory '" , toFilePath parentDir @@ -1209,7 +1230,7 @@ -- | Get the URL to request the information on the latest snapshots askLatestSnapshotUrl :: (MonadReader env m, HasConfig env) => m Text -askLatestSnapshotUrl = asks (configLatestSnapshotUrl . getConfig) +askLatestSnapshotUrl = asks (urlsLatestSnapshot . configUrls . getConfig) -- | Root for a specific package index configPackageIndexRoot :: (MonadReader env m, HasConfig env, MonadThrow m) => IndexName -> m (Path Abs Dir) @@ -1244,15 +1265,15 @@ getWorkDir = configWorkDir `liftM` asks getConfig -- | Per-project work dir -configProjectWorkDir :: (HasBuildConfig env, MonadReader env m) => m (Path Abs Dir) -configProjectWorkDir = do +getProjectWorkDir :: (HasBuildConfig env, MonadReader env m) => m (Path Abs Dir) +getProjectWorkDir = do bc <- asks getBuildConfig workDir <- getWorkDir return (bcRoot bc workDir) -- | File containing the installed cache, see "Stack.PackageDump" configInstalledCache :: (HasBuildConfig env, MonadReader env m) => m (Path Abs File) -configInstalledCache = liftM ( $(mkRelFile "installed-cache.bin")) configProjectWorkDir +configInstalledCache = liftM ( $(mkRelFile "installed-cache.bin")) getProjectWorkDir -- | Relative directory for the platform identifier platformOnlyRelDir @@ -1283,7 +1304,7 @@ installationRootLocal = do bc <- asks getBuildConfig psc <- useShaPathOnWindows =<< platformSnapAndCompilerRel - return $ configProjectWorkDir bc $(mkRelDir "install") psc + return $ getProjectWorkDir bc $(mkRelDir "install") psc -- | Path for platform followed by snapshot name followed by compiler -- name. @@ -1409,6 +1430,19 @@ getWhichCompiler :: (MonadReader env m, HasEnvConfig env) => m WhichCompiler getWhichCompiler = asks (whichCompiler . envConfigCompilerVersion . getEnvConfig) +-- | Get the path for the given compiler ignoring any local binaries. +-- +-- https://github.com/commercialhaskell/stack/issues/1052 +getCompilerPath + :: (MonadIO m, MonadThrow m, MonadReader env m, HasConfig env) + => WhichCompiler + -> m (Path Abs File) +getCompilerPath wc = do + config <- asks getConfig + eoWithoutLocals <- liftIO $ + configEnvOverride config minimalEnvSettings { esLocaleUtf8 = True } + join (findExecutable eoWithoutLocals (compilerExeName wc)) + data ProjectAndConfigMonoid = ProjectAndConfigMonoid !Project !ConfigMonoid @@ -1458,8 +1492,7 @@ -- | A PackageEntry for the current directory, used as a default packageEntryCurrDir :: PackageEntry packageEntryCurrDir = PackageEntry - { peValidWanted = Nothing - , peExtraDepMaybe = Nothing + { peExtraDep = False , peLocation = PLFilePath "." , peSubdirs = [] } diff --git a/src/Stack/Types/Config.hs-boot b/src/Stack/Types/Config.hs-boot deleted file mode 100644 --- a/src/Stack/Types/Config.hs-boot +++ /dev/null @@ -1,5 +0,0 @@ -module Stack.Types.Config where - -data AbstractResolver -data Resolver -data Config diff --git a/src/Stack/Types/Config/Build.hs b/src/Stack/Types/Config/Build.hs new file mode 100644 --- /dev/null +++ b/src/Stack/Types/Config/Build.hs @@ -0,0 +1,366 @@ +{-# LANGUAGE FlexibleInstances, RecordWildCards, OverloadedStrings #-} + +-- | Configuration options for building. + +module Stack.Types.Config.Build + ( + BuildOpts(..) + , BuildCommand(..) + , defaultBuildOpts + , defaultBuildOptsCLI + , BuildOptsCLI(..) + , BuildOptsMonoid(..) + , TestOpts(..) + , defaultTestOpts + , TestOptsMonoid(..) + , BenchmarkOpts(..) + , defaultBenchmarkOpts + , BenchmarkOptsMonoid(..) + , FileWatchOpts(..) + , BuildSubset(..) + ) + where + +import Data.Aeson.Extended +import Data.Map.Strict (Map) +import qualified Data.Map.Strict as Map +import Data.Monoid +import Data.Text (Text) +import Stack.Types.FlagName +import Stack.Types.PackageName +import Control.Applicative + +-- | Build options that is interpreted by the build command. +-- This is built up from BuildOptsCLI and BuildOptsMonoid +data BuildOpts = + BuildOpts {boptsLibProfile :: !Bool + ,boptsExeProfile :: !Bool + ,boptsHaddock :: !Bool + -- ^ Build haddocks? + ,boptsOpenHaddocks :: !Bool + -- ^ Open haddocks in the browser? + ,boptsHaddockDeps :: !(Maybe Bool) + -- ^ Build haddocks for dependencies? + ,boptsInstallExes :: !Bool + -- ^ Install executables to user path after building? + ,boptsPreFetch :: !Bool + -- ^ Fetch all packages immediately + -- ^ Watch files for changes and automatically rebuild + ,boptsKeepGoing :: !(Maybe Bool) + -- ^ Keep building/running after failure + ,boptsForceDirty :: !Bool + -- ^ Force treating all local packages as having dirty files + + ,boptsTests :: !Bool + -- ^ Turn on tests for local targets + ,boptsTestOpts :: !TestOpts + -- ^ Additional test arguments + + ,boptsBenchmarks :: !Bool + -- ^ Turn on benchmarks for local targets + ,boptsBenchmarkOpts :: !BenchmarkOpts + -- ^ Additional test arguments + -- ^ Commands (with arguments) to run after a successful build + -- ^ Only perform the configure step when building + ,boptsReconfigure :: !Bool + -- ^ Perform the configure step even if already configured + ,boptsCabalVerbose :: !Bool + -- ^ Ask Cabal to be verbose in its builds + ,boptsSplitObjs :: !Bool + -- ^ Whether to enable split-objs. + } + deriving (Show) + +defaultBuildOpts :: BuildOpts +defaultBuildOpts = BuildOpts + { boptsLibProfile = False + , boptsExeProfile = False + , boptsHaddock = False + , boptsOpenHaddocks = False + , boptsHaddockDeps = Nothing + , boptsInstallExes = False + , boptsPreFetch = False + , boptsKeepGoing = Nothing + , boptsForceDirty = False + , boptsTests = False + , boptsTestOpts = defaultTestOpts + , boptsBenchmarks = False + , boptsBenchmarkOpts = defaultBenchmarkOpts + , boptsReconfigure = False + , boptsCabalVerbose = False + , boptsSplitObjs = False + } + +defaultBuildOptsCLI ::BuildOptsCLI +defaultBuildOptsCLI = BuildOptsCLI + { boptsCLITargets = [] + , boptsCLIDryrun = False + , boptsCLIFlags = Map.empty + , boptsCLIGhcOptions = [] + , boptsCLIBuildSubset = BSAll + , boptsCLIFileWatch = NoFileWatch + , boptsCLIExec = [] + , boptsCLIOnlyConfigure = False + , boptsCLICommand = Build + } + +-- | Build options that may only be specified from the CLI +data BuildOptsCLI = BuildOptsCLI + { boptsCLITargets :: ![Text] + , boptsCLIDryrun :: !Bool + , boptsCLIGhcOptions :: ![Text] + , boptsCLIFlags :: !(Map (Maybe PackageName) (Map FlagName Bool)) + , boptsCLIBuildSubset :: !BuildSubset + , boptsCLIFileWatch :: !FileWatchOpts + , boptsCLIExec :: ![(String, [String])] + , boptsCLIOnlyConfigure :: !Bool + , boptsCLICommand :: !BuildCommand + } deriving Show + +-- | Command sum type for conditional arguments. +data BuildCommand + = Build + | Test + | Haddock + | Bench + | Install + deriving (Eq, Show) + +-- | Build options that may be specified in the stack.yaml or from the CLI +data BuildOptsMonoid = BuildOptsMonoid + { buildMonoidLibProfile :: !(Maybe Bool) + , buildMonoidExeProfile :: !(Maybe Bool) + , buildMonoidHaddock :: !(Maybe Bool) + , buildMonoidOpenHaddocks :: !(Maybe Bool) + , buildMonoidHaddockDeps :: !(Maybe Bool) + , buildMonoidInstallExes :: !(Maybe Bool) + , buildMonoidPreFetch :: !(Maybe Bool) + , buildMonoidKeepGoing :: !(Maybe Bool) + , buildMonoidForceDirty :: !(Maybe Bool) + , buildMonoidTests :: !(Maybe Bool) + , buildMonoidTestOpts :: !TestOptsMonoid + , buildMonoidBenchmarks :: !(Maybe Bool) + , buildMonoidBenchmarkOpts :: !BenchmarkOptsMonoid + , buildMonoidReconfigure :: !(Maybe Bool) + , buildMonoidCabalVerbose :: !(Maybe Bool) + , buildMonoidSplitObjs :: !(Maybe Bool) + } deriving (Show) + +instance FromJSON (WithJSONWarnings BuildOptsMonoid) where + parseJSON = withObjectWarnings "BuildOptsMonoid" + (\o -> do buildMonoidLibProfile <- o ..:? buildMonoidLibProfileArgName + buildMonoidExeProfile <- o ..:? buildMonoidExeProfileArgName + buildMonoidHaddock <- o ..:? buildMonoidHaddockArgName + buildMonoidOpenHaddocks <- o ..:? buildMonoidOpenHaddocksArgName + buildMonoidHaddockDeps <- o ..:? buildMonoidHaddockDepsArgName + buildMonoidInstallExes <- o ..:? buildMonoidInstallExesArgName + buildMonoidPreFetch <- o ..:? buildMonoidPreFetchArgName + buildMonoidKeepGoing <- o ..:? buildMonoidKeepGoingArgName + buildMonoidForceDirty <- o ..:? buildMonoidForceDirtyArgName + buildMonoidTests <- o ..:? buildMonoidTestsArgName + buildMonoidTestOpts <- jsonSubWarnings (o ..:? buildMonoidTestOptsArgName ..!= mempty) + buildMonoidBenchmarks <- o ..:? buildMonoidBenchmarksArgName + buildMonoidBenchmarkOpts <- jsonSubWarnings (o ..:? buildMonoidBenchmarkOptsArgName ..!= mempty) + buildMonoidReconfigure <- o ..:? buildMonoidReconfigureArgName + buildMonoidCabalVerbose <- o ..:? buildMonoidCabalVerboseArgName + buildMonoidSplitObjs <- o ..:? buildMonoidSplitObjsName + return BuildOptsMonoid{..}) + +buildMonoidLibProfileArgName :: Text +buildMonoidLibProfileArgName = "library-profiling" + +buildMonoidExeProfileArgName :: Text +buildMonoidExeProfileArgName = "executable-profiling" + +buildMonoidHaddockArgName :: Text +buildMonoidHaddockArgName = "haddock" + +buildMonoidOpenHaddocksArgName :: Text +buildMonoidOpenHaddocksArgName = "open-haddocks" + +buildMonoidHaddockDepsArgName :: Text +buildMonoidHaddockDepsArgName = "haddock-deps" + +buildMonoidInstallExesArgName :: Text +buildMonoidInstallExesArgName = "copy-bins" + +buildMonoidPreFetchArgName :: Text +buildMonoidPreFetchArgName = "prefetch" + +buildMonoidKeepGoingArgName :: Text +buildMonoidKeepGoingArgName = "keep-going" + +buildMonoidForceDirtyArgName :: Text +buildMonoidForceDirtyArgName = "force-dirty" + +buildMonoidTestsArgName :: Text +buildMonoidTestsArgName = "test" + +buildMonoidTestOptsArgName :: Text +buildMonoidTestOptsArgName = "test-arguments" + +buildMonoidBenchmarksArgName :: Text +buildMonoidBenchmarksArgName = "bench" + +buildMonoidBenchmarkOptsArgName :: Text +buildMonoidBenchmarkOptsArgName = "benchmark-opts" + +buildMonoidReconfigureArgName :: Text +buildMonoidReconfigureArgName = "reconfigure" + +buildMonoidCabalVerboseArgName :: Text +buildMonoidCabalVerboseArgName = "cabal-verbose" + +buildMonoidSplitObjsName :: Text +buildMonoidSplitObjsName = "split-objs" + +instance Monoid BuildOptsMonoid where + mempty = BuildOptsMonoid + {buildMonoidLibProfile = Nothing + ,buildMonoidExeProfile = Nothing + ,buildMonoidHaddock = Nothing + ,buildMonoidOpenHaddocks = Nothing + ,buildMonoidHaddockDeps = Nothing + ,buildMonoidInstallExes = Nothing + ,buildMonoidPreFetch = Nothing + ,buildMonoidKeepGoing = Nothing + ,buildMonoidForceDirty = Nothing + ,buildMonoidTests = Nothing + ,buildMonoidTestOpts = mempty + ,buildMonoidBenchmarks = Nothing + ,buildMonoidBenchmarkOpts = mempty + ,buildMonoidReconfigure = Nothing + ,buildMonoidCabalVerbose = Nothing + ,buildMonoidSplitObjs = Nothing + } + + mappend l r = BuildOptsMonoid + {buildMonoidLibProfile = buildMonoidLibProfile l <|> buildMonoidLibProfile r + ,buildMonoidExeProfile = buildMonoidExeProfile l <|> buildMonoidExeProfile r + ,buildMonoidHaddock = buildMonoidHaddock l <|> buildMonoidHaddock r + ,buildMonoidOpenHaddocks = buildMonoidOpenHaddocks l <|> buildMonoidOpenHaddocks r + ,buildMonoidHaddockDeps = buildMonoidHaddockDeps l <|> buildMonoidHaddockDeps r + ,buildMonoidInstallExes = buildMonoidInstallExes l <|> buildMonoidInstallExes r + ,buildMonoidPreFetch = buildMonoidPreFetch l <|> buildMonoidPreFetch r + ,buildMonoidKeepGoing = buildMonoidKeepGoing l <|> buildMonoidKeepGoing r + ,buildMonoidForceDirty = buildMonoidForceDirty l <|> buildMonoidForceDirty r + ,buildMonoidTests = buildMonoidTests l <|> buildMonoidTests r + ,buildMonoidTestOpts = buildMonoidTestOpts l <> buildMonoidTestOpts r + ,buildMonoidBenchmarks = buildMonoidBenchmarks l <|> buildMonoidBenchmarks r + ,buildMonoidBenchmarkOpts = buildMonoidBenchmarkOpts l <> buildMonoidBenchmarkOpts r + ,buildMonoidReconfigure = buildMonoidReconfigure l <|> buildMonoidReconfigure r + ,buildMonoidCabalVerbose = buildMonoidCabalVerbose l <|> buildMonoidCabalVerbose r + ,buildMonoidSplitObjs = buildMonoidSplitObjs l <|> buildMonoidSplitObjs r + } + +-- | Which subset of packages to build +data BuildSubset + = BSAll + | BSOnlySnapshot + -- ^ Only install packages in the snapshot database, skipping + -- packages intended for the local database. + | BSOnlyDependencies + deriving (Show, Eq) + +-- | Options for the 'FinalAction' 'DoTests' +data TestOpts = + TestOpts {toRerunTests :: !Bool -- ^ Whether successful tests will be run gain + ,toAdditionalArgs :: ![String] -- ^ Arguments passed to the test program + ,toCoverage :: !Bool -- ^ Generate a code coverage report + ,toDisableRun :: !Bool -- ^ Disable running of tests + } deriving (Eq,Show) + +defaultTestOpts :: TestOpts +defaultTestOpts = TestOpts + { toRerunTests = True + , toAdditionalArgs = [] + , toCoverage = False + , toDisableRun = False + } + +data TestOptsMonoid = + TestOptsMonoid + { toMonoidRerunTests :: !(Maybe Bool) + , toMonoidAdditionalArgs :: ![String] + , toMonoidCoverage :: !(Maybe Bool) + , toMonoidDisableRun :: !(Maybe Bool) + } deriving (Show) + +instance FromJSON (WithJSONWarnings TestOptsMonoid) where + parseJSON = withObjectWarnings "TestOptsMonoid" + (\o -> do toMonoidRerunTests <- o ..:? toMonoidRerunTestsArgName + toMonoidAdditionalArgs <- o ..:? toMonoidAdditionalArgsName ..!= [] + toMonoidCoverage <- o ..:? toMonoidCoverageArgName + toMonoidDisableRun <- o ..:? toMonoidDisableRunArgName + return TestOptsMonoid{..}) + +toMonoidRerunTestsArgName :: Text +toMonoidRerunTestsArgName = "rerun-tests" + +toMonoidAdditionalArgsName :: Text +toMonoidAdditionalArgsName = "additional-args" + +toMonoidCoverageArgName :: Text +toMonoidCoverageArgName = "coverage" + +toMonoidDisableRunArgName :: Text +toMonoidDisableRunArgName = "no-run-tests" + +instance Monoid TestOptsMonoid where + mempty = TestOptsMonoid + { toMonoidRerunTests = Nothing + , toMonoidAdditionalArgs = [] + , toMonoidCoverage = Nothing + , toMonoidDisableRun = Nothing + } + mappend l r = TestOptsMonoid + { toMonoidRerunTests = toMonoidRerunTests l <|> toMonoidRerunTests r + , toMonoidAdditionalArgs = toMonoidAdditionalArgs l <> toMonoidAdditionalArgs r + , toMonoidCoverage = toMonoidCoverage l <|> toMonoidCoverage r + , toMonoidDisableRun = toMonoidDisableRun l <|> toMonoidDisableRun r + } + +-- | Options for the 'FinalAction' 'DoBenchmarks' +data BenchmarkOpts = + BenchmarkOpts + { beoAdditionalArgs :: !(Maybe String) -- ^ Arguments passed to the benchmark program + , beoDisableRun :: !Bool -- ^ Disable running of benchmarks + } deriving (Eq,Show) + +defaultBenchmarkOpts :: BenchmarkOpts +defaultBenchmarkOpts = BenchmarkOpts + { beoAdditionalArgs = Nothing + , beoDisableRun = False + } + +data BenchmarkOptsMonoid = + BenchmarkOptsMonoid + { beoMonoidAdditionalArgs :: !(Maybe String) + , beoMonoidDisableRun :: !(Maybe Bool) + } deriving (Show) + +instance FromJSON (WithJSONWarnings BenchmarkOptsMonoid) where + parseJSON = withObjectWarnings "BenchmarkOptsMonoid" + (\o -> do beoMonoidAdditionalArgs <- o ..:? beoMonoidAdditionalArgsArgName + beoMonoidDisableRun <- o ..:? beoMonoidDisableRunArgName + return BenchmarkOptsMonoid{..}) + +beoMonoidAdditionalArgsArgName :: Text +beoMonoidAdditionalArgsArgName = "benchmark-arguments" + +beoMonoidDisableRunArgName :: Text +beoMonoidDisableRunArgName = "no-run-benchmarks" + +instance Monoid BenchmarkOptsMonoid where + mempty = BenchmarkOptsMonoid + { beoMonoidAdditionalArgs = Nothing + , beoMonoidDisableRun = Nothing} + mappend l r = BenchmarkOptsMonoid + { beoMonoidAdditionalArgs = beoMonoidAdditionalArgs l <|> beoMonoidAdditionalArgs r + , beoMonoidDisableRun = beoMonoidDisableRun l <|> beoMonoidDisableRun r} + +data FileWatchOpts + = NoFileWatch + | FileWatch + | FileWatchPoll + deriving (Show,Eq) diff --git a/src/Stack/Types/Internal.hs b/src/Stack/Types/Internal.hs --- a/src/Stack/Types/Internal.hs +++ b/src/Stack/Types/Internal.hs @@ -1,3 +1,5 @@ +{-# LANGUAGE Rank2Types #-} + -- | Internal types to the library. module Stack.Types.Internal where @@ -5,6 +7,7 @@ import Control.Concurrent.MVar import Control.Monad.Logger (LogLevel) import Data.Text (Text) +import Lens.Micro import Network.HTTP.Client.Conduit (Manager,HasHttpManager(..)) import Stack.Types.Config @@ -71,3 +74,55 @@ instance HasSticky (Env config) where getSticky = envSticky + +envEnvConfig :: Lens' (Env EnvConfig) EnvConfig +envEnvConfig = lens (envConfig) + (\s t -> s {envConfig = t}) + +buildOptsMonoidHaddock :: Lens' BuildOptsMonoid (Maybe Bool) +buildOptsMonoidHaddock = lens (buildMonoidHaddock) + (\buildMonoid t -> buildMonoid {buildMonoidHaddock = t}) + +buildOptsMonoidTests :: Lens' BuildOptsMonoid (Maybe Bool) +buildOptsMonoidTests = lens (buildMonoidTests) + (\buildMonoid t -> buildMonoid {buildMonoidTests = t}) + +buildOptsMonoidBenchmarks :: Lens' BuildOptsMonoid (Maybe Bool) +buildOptsMonoidBenchmarks = lens (buildMonoidBenchmarks) + (\buildMonoid t -> buildMonoid {buildMonoidBenchmarks = t}) + +buildOptsMonoidInstallExes :: Lens' BuildOptsMonoid (Maybe Bool) +buildOptsMonoidInstallExes = + lens (buildMonoidInstallExes) + (\buildMonoid t -> buildMonoid {buildMonoidInstallExes = t}) + +buildOptsInstallExes :: Lens' BuildOpts Bool +buildOptsInstallExes = + lens (boptsInstallExes) + (\bopts t -> bopts {boptsInstallExes = t}) + +envConfigBuildOpts :: Lens' EnvConfig BuildOpts +envConfigBuildOpts = + lens + (\envCfg -> configBuild (bcConfig (envConfigBuildConfig envCfg))) + (\envCfg bopts -> + envCfg + { envConfigBuildConfig = (envConfigBuildConfig envCfg) + { bcConfig = (bcConfig (envConfigBuildConfig envCfg)) + { configBuild = bopts + } + } + }) + +globalOptsBuildOptsMonoid :: Lens' GlobalOpts BuildOptsMonoid +globalOptsBuildOptsMonoid = + lens + (\globalOpts -> + configMonoidBuildOpts + (globalConfigMonoid globalOpts)) + (\globalOpts boptsMonoid -> + globalOpts + { globalConfigMonoid = (globalConfigMonoid globalOpts) + { configMonoidBuildOpts = boptsMonoid + } + }) diff --git a/src/Stack/Types/Nix.hs b/src/Stack/Types/Nix.hs --- a/src/Stack/Types/Nix.hs +++ b/src/Stack/Types/Nix.hs @@ -11,9 +11,6 @@ import Data.Text (Text) import Data.Monoid import Prelude -import Stack.Types.Compiler (CompilerVersion) -import {-# SOURCE #-} Stack.Types.Config (Resolver) -import Text.Show.Functions () -- | Nix configuration. Parameterize by resolver type to avoid cyclic -- dependency. @@ -26,8 +23,6 @@ -- ^ The path of a file containing preconfiguration of the environment (e.g shell.nix) ,nixShellOptions :: ![Text] -- ^ Options to be given to the nix-shell command line - ,nixCompiler :: !(Maybe Resolver -> Maybe CompilerVersion -> Text) - -- ^ Yield a compiler attribute name given a resolver override. } deriving (Show) @@ -54,7 +49,7 @@ -- | Decode uninterpreted nix options from JSON/YAML. instance FromJSON (WithJSONWarnings NixOptsMonoid) where parseJSON = withObjectWarnings "NixOptsMonoid" - (\o -> do nixMonoidDefaultEnable <- pure True + (\o -> do nixMonoidDefaultEnable <- pure False nixMonoidEnable <- o ..:? nixEnableArgName nixMonoidPureShell <- o ..:? nixPureShellArgName nixMonoidPackages <- o ..:? nixPackagesArgName diff --git a/src/Stack/Types/Package.hs b/src/Stack/Types/Package.hs --- a/src/Stack/Types/Package.hs +++ b/src/Stack/Types/Package.hs @@ -19,6 +19,7 @@ import Data.Data import Data.Function import Data.List +import qualified Data.Map as M import Data.Map.Strict (Map) import Data.Maybe import Data.Monoid @@ -33,15 +34,15 @@ import Distribution.PackageDescription (TestSuiteInterface) import Distribution.System (Platform (..)) import Distribution.Text (display) -import GHC.Generics +import GHC.Generics (Generic) import Path as FL import Prelude import Stack.Types.Compiler import Stack.Types.Config import Stack.Types.FlagName import Stack.Types.GhcPkgId -import Stack.Types.PackageName import Stack.Types.PackageIdentifier +import Stack.Types.PackageName import Stack.Types.Version -- | All exceptions thrown by the library. @@ -87,6 +88,7 @@ ,packageTools :: ![Dependency] -- ^ A build tool name. ,packageAllDeps :: !(Set PackageName) -- ^ Original dependencies (not sieved). ,packageFlags :: !(Map FlagName Bool) -- ^ Flags used on package. + ,packageDefaultFlags :: !(Map FlagName Bool) -- ^ Defaults for unspecified flags. ,packageHasLibrary :: !Bool -- ^ does the package have a buildable library stanza? ,packageTests :: !(Map Text TestSuiteInterface) -- ^ names and interfaces of test suites ,packageBenchmarks :: !(Set Text) -- ^ names of benchmarks @@ -94,7 +96,6 @@ ,packageOpts :: !GetPackageOpts -- ^ Args to pass to GHC. ,packageHasExposedModules :: !Bool -- ^ Does the package have exposed modules? ,packageSimpleType :: !Bool -- ^ Does the package of build-type: Simple - ,packageDefinedFlags :: !(Set FlagName) -- ^ All flags defined in the .cabal file } deriving (Show,Typeable) @@ -102,6 +103,9 @@ packageIdentifier pkg = PackageIdentifier (packageName pkg) (packageVersion pkg) +packageDefinedFlags :: Package -> Set FlagName +packageDefinedFlags = M.keysSet . packageDefaultFlags + -- | Files that the package depends on, relative to package directory. -- Argument is the location of the .cabal file newtype GetPackageOpts = GetPackageOpts @@ -151,6 +155,8 @@ data PackageWarning = UnlistedModulesWarning (Path Abs File) (Maybe String) [ModuleName] -- ^ Modules found that are not listed in cabal file + | MissingModulesWarning (Path Abs File) (Maybe String) [ModuleName] + -- ^ Modules not found in file system, which are listed in cabal file instance Show PackageWarning where show (UnlistedModulesWarning cabalfp component [unlistedModule]) = concat @@ -170,7 +176,26 @@ Just c -> " for '" ++ c ++ "'" , " component (add to other-modules):\n " , intercalate "\n " (map display unlistedModules)] + show (MissingModulesWarning cabalfp component [missingModule]) = + concat + [ "module listed in " + , toFilePath (filename cabalfp) + , case component of + Nothing -> " for library" + Just c -> " for '" ++ c ++ "'" + , " component not found in filesystem: " + , display missingModule] + show (MissingModulesWarning cabalfp component missingModules) = + concat + [ "modules listed in " + , toFilePath (filename cabalfp) + , case component of + Nothing -> " for library" + Just c -> " for '" ++ c ++ "'" + , " component not found in filesystem:\n " + , intercalate "\n " (map display missingModules)] + -- | Package build configuration data PackageConfig = PackageConfig {packageConfigEnableTests :: !Bool -- ^ Are tests enabled? @@ -236,6 +261,7 @@ -- ^ Directory of the package. , lpCabalFile :: !(Path Abs File) -- ^ The .cabal file + , lpForceDirty :: !Bool , lpDirtyFiles :: !(Maybe (Set FilePath)) -- ^ Nothing == not dirty, Just == dirty. Note that the Set may be empty if -- we forced the build to treat packages as dirty. Also, the Set may not @@ -392,7 +418,10 @@ data Installed = Library PackageIdentifier GhcPkgId | Executable PackageIdentifier deriving (Show, Eq, Ord) +installedPackageIdentifier :: Installed -> PackageIdentifier +installedPackageIdentifier (Library pid _) = pid +installedPackageIdentifier (Executable pid) = pid + -- | Get the installed Version. installedVersion :: Installed -> Version -installedVersion (Library (PackageIdentifier _ v) _) = v -installedVersion (Executable (PackageIdentifier _ v)) = v +installedVersion = packageIdentifierVersion . installedPackageIdentifier diff --git a/src/Stack/Types/PackageName.hs b/src/Stack/Types/PackageName.hs --- a/src/Stack/Types/PackageName.hs +++ b/src/Stack/Types/PackageName.hs @@ -164,4 +164,8 @@ p s = case parsePackageNameFromString s of Just x -> Right x - Nothing -> Left ("Expected valid package name, but got: " ++ s) + Nothing -> Left $ unlines + [ "Expected valid package name, but got: " ++ s + , "Package names consist of one or more alphanumeric words separated by hyphens." + , "To avoid ambiguity with version numbers, each of these words must contain at least one letter." + ] diff --git a/src/Stack/Types/Sig.hs b/src/Stack/Types/Sig.hs --- a/src/Stack/Types/Sig.hs +++ b/src/Stack/Types/Sig.hs @@ -1,7 +1,7 @@ +{-# LANGUAGE CPP #-} {-# LANGUAGE DeriveDataTypeable #-} {-# LANGUAGE FlexibleInstances #-} {-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE RecordWildCards #-} {-| Module : Stack.Types.Sig @@ -14,14 +14,17 @@ -} module Stack.Types.Sig - (Signature(..), Fingerprint(..), SigException(..)) - where + (Signature(..), Fingerprint, mkFingerprint, SigException(..)) where +#if __GLASGOW_HASKELL__ < 710 +import Control.Applicative ((<$>)) +#endif + import Control.Exception (Exception) import Data.Aeson (Value(..), ToJSON(..), FromJSON(..)) import Data.ByteString (ByteString) import qualified Data.ByteString as SB -import Data.Char (isDigit, isAlpha, isSpace) +import Data.Char (isHexDigit) import Data.Monoid ((<>)) import Data.String (IsString(..)) import Data.Text (Text) @@ -42,27 +45,27 @@ else show (SB.take 140 s)) -- | The GPG fingerprint. -newtype Fingerprint = Fingerprint - { fingerprintSample :: Text - } deriving (Eq,Ord,Show) +newtype Fingerprint = + Fingerprint Text + deriving (Eq,Ord) +mkFingerprint :: Text -> Fingerprint +mkFingerprint = Fingerprint . hexText + +hexText :: Text -> Text +hexText = T.toUpper . T.dropWhile (not . isHexDigit) + +instance Show Fingerprint where + show (Fingerprint hex) = T.unpack (hexText hex) + instance FromJSON Fingerprint where - parseJSON j = do - s <- parseJSON j - let withoutSpaces = T.filter (not . isSpace) s - if T.null withoutSpaces || - T.all - (\c -> - isAlpha c || isDigit c || isSpace c) - withoutSpaces - then return (Fingerprint withoutSpaces) - else fail ("Expected fingerprint, but got: " ++ T.unpack s) + parseJSON j = Fingerprint . hexText <$> parseJSON j instance ToJSON Fingerprint where - toJSON (Fingerprint txt) = String txt + toJSON (Fingerprint hex) = String (hexText hex) instance IsString Fingerprint where - fromString = Fingerprint . T.pack + fromString = Fingerprint . hexText . T.pack instance FromJSON (Aeson PackageName) where parseJSON j = do @@ -79,6 +82,7 @@ -- | Exceptions data SigException = GPGFingerprintException String + | GPGNotFoundException | GPGSignException String | GPGVerifyException String | SigInvalidSDistTarBall @@ -91,6 +95,7 @@ instance Show SigException where show (GPGFingerprintException e) = "Error extracting a GPG fingerprint " <> e + show GPGNotFoundException = "Unable to find gpg2 or gpg executable" show (GPGSignException e) = "Error signing with GPG " <> e show (GPGVerifyException e) = "Error verifying with GPG " <> e show SigNoProjectRootException = "Missing Project Root" diff --git a/src/Stack/Types/TemplateName.hs b/src/Stack/Types/TemplateName.hs --- a/src/Stack/Types/TemplateName.hs +++ b/src/Stack/Types/TemplateName.hs @@ -10,10 +10,12 @@ import Control.Error.Safe (justErr) import Control.Applicative import Data.Aeson.Extended (FromJSON, withText, parseJSON) +import Data.Aeson.Types (typeMismatch) import Data.Foldable (asum) import Data.Monoid import Data.Text (Text) import qualified Data.Text as T +import Data.Yaml (Value(Object), (.:?)) import Language.Haskell.TH import Network.HTTP.Client (parseUrl) import qualified Options.Applicative as O @@ -37,6 +39,15 @@ instance FromJSON TemplateName where parseJSON = withText "TemplateName" $ either fail return . parseTemplateNameFromString . T.unpack + +data TemplateInfo = TemplateInfo + { author :: Maybe Text + , description :: Maybe Text } + deriving (Eq, Ord, Show) + +instance FromJSON TemplateInfo where + parseJSON (Object v) = TemplateInfo <$> v .:? "author" <*> v .:? "description" + parseJSON invalid = typeMismatch "Template Info" invalid -- | An argument which accepts a template name of the format -- @foo.hsfiles@ or @foo@, ultimately normalized to @foo@. diff --git a/src/Stack/Types/Urls.hs b/src/Stack/Types/Urls.hs new file mode 100644 --- /dev/null +++ b/src/Stack/Types/Urls.hs @@ -0,0 +1,46 @@ +{-# LANGUAGE OverloadedStrings, FlexibleInstances #-} + +module Stack.Types.Urls where + +import Control.Applicative +import Data.Aeson.Extended +import Data.Text (Text) +import Data.Monoid +import Prelude + +data Urls = Urls + { urlsLatestSnapshot :: !Text + , urlsLtsBuildPlans :: !Text + , urlsNightlyBuildPlans :: !Text + } + deriving Show + +-- TODO: Really need this instance? +instance FromJSON (WithJSONWarnings Urls) where + parseJSON = withObjectWarnings "Urls" $ \o -> do + Urls + <$> o ..: "latest-snapshot" + <*> o ..: "lts-build-plans" + <*> o ..: "nightly-build-plans" + +data UrlsMonoid = UrlsMonoid + { urlsMonoidLatestSnapshot :: !(Maybe Text) + , urlsMonoidLtsBuildPlans :: !(Maybe Text) + , urlsMonoidNightlyBuildPlans :: !(Maybe Text) + } + deriving Show + +instance FromJSON (WithJSONWarnings UrlsMonoid) where + parseJSON = withObjectWarnings "UrlsMonoid" $ \o -> do + UrlsMonoid + <$> o ..: "latest-snapshot" + <*> o ..: "lts-build-plans" + <*> o ..: "nightly-build-plans" + +instance Monoid UrlsMonoid where + mempty = UrlsMonoid Nothing Nothing Nothing + mappend l r = UrlsMonoid + { urlsMonoidLatestSnapshot = urlsMonoidLatestSnapshot l <|> urlsMonoidLatestSnapshot r + , urlsMonoidLtsBuildPlans = urlsMonoidLtsBuildPlans l <|> urlsMonoidLtsBuildPlans r + , urlsMonoidNightlyBuildPlans = urlsMonoidNightlyBuildPlans l <|> urlsMonoidNightlyBuildPlans r + } diff --git a/src/Stack/Upgrade.hs b/src/Stack/Upgrade.hs --- a/src/Stack/Upgrade.hs +++ b/src/Stack/Upgrade.hs @@ -17,12 +17,12 @@ import qualified Data.Monoid import qualified Data.Set as Set import qualified Data.Text as T +import Lens.Micro (set) import Network.HTTP.Client.Conduit (HasHttpManager) import Path import Path.IO import qualified Paths_stack as Paths import Stack.Build -import Stack.Types.Build import Stack.Config import Stack.Fetch import Stack.PackageIndex @@ -65,7 +65,7 @@ return $ Just $ tmp $(mkRelDir "stack") Nothing -> do updateAllIndices menv - caches <- getPackageCaches menv + caches <- getPackageCaches let latest = Map.fromListWith max $ map toTuple $ Map.keys @@ -101,8 +101,7 @@ envConfig1 <- runInnerStackT bconfig $ setupEnv $ Just $ "Try rerunning with --install-ghc to install the correct GHC into " <> T.pack (toFilePath (configLocalPrograms config)) - runInnerStackT envConfig1 $ - build (const $ return ()) Nothing defaultBuildOpts - { boptsTargets = ["stack"] - , boptsInstallExes = True + runInnerStackT (set (envConfigBuildOpts.buildOptsInstallExes) True envConfig1) $ + build (const $ return ()) Nothing defaultBuildOptsCLI + { boptsCLITargets = ["stack"] } diff --git a/src/System/Process/Log.hs b/src/System/Process/Log.hs --- a/src/System/Process/Log.hs +++ b/src/System/Process/Log.hs @@ -3,7 +3,8 @@ -- | Separate module because TH. module System.Process.Log - (logProcessRun + (logCreateProcess + ,logProcessRun ,showProcessArgDebug) where @@ -12,6 +13,21 @@ import Data.Text (Text) import qualified Data.Text as T import Language.Haskell.TH +import System.Process (CreateProcess(..), CmdSpec(..)) + +-- | Log running a process with its arguments, for debugging (-v). +logCreateProcess :: Q Exp +logCreateProcess = + [|let f :: MonadLogger m => CreateProcess -> m () + f (CreateProcess { cmdspec = ShellCommand shellCmd }) = + $logDebug ("Creating shell process: " <> T.pack shellCmd) + f (CreateProcess { cmdspec = RawCommand name args }) = + $logDebug + ("Creating process: " <> T.pack name <> " " <> + T.intercalate + " " + (map showProcessArgDebug args)) + in f|] -- | Log running a process with its arguments, for debugging (-v). logProcessRun :: Q Exp diff --git a/src/System/Process/Read.hs b/src/System/Process/Read.hs --- a/src/System/Process/Read.hs +++ b/src/System/Process/Read.hs @@ -15,6 +15,7 @@ ,sinkProcessStdout ,sinkProcessStderrStdout ,sinkProcessStderrStdoutHandle + ,logProcessStderrStdout ,readProcess ,EnvOverride(..) ,unEnvOverride @@ -38,15 +39,16 @@ import Control.Arrow ((***), first) import Control.Concurrent.Async (concurrently) import Control.Exception hiding (try, catch) -import Control.Monad (join, liftM, unless) +import Control.Monad (join, liftM, unless, void) import Control.Monad.Catch (MonadThrow, MonadCatch, throwM, try, catch) import Control.Monad.IO.Class (MonadIO, liftIO) -import Control.Monad.Logger (MonadLogger, logError) -import Control.Monad.Trans.Control (MonadBaseControl) +import Control.Monad.Logger +import Control.Monad.Trans.Control (MonadBaseControl, liftBaseWith) import qualified Data.ByteString as S -import qualified Data.ByteString.Lazy as L import Data.ByteString.Builder +import qualified Data.ByteString.Lazy as L import Data.Conduit +import qualified Data.Conduit.Binary as CB import qualified Data.Conduit.List as CL import Data.Conduit.Process hiding (callProcess) import Data.Foldable (forM_) @@ -58,10 +60,11 @@ import Data.Text (Text) import qualified Data.Text as T import Data.Text.Encoding.Error (lenientDecode) -import qualified Data.Text.Lazy.Encoding as LT import qualified Data.Text.Lazy as LT +import qualified Data.Text.Lazy.Encoding as LT import Data.Typeable (Typeable) import Distribution.System (OS (Windows), Platform (Platform)) +import Language.Haskell.TH as TH (location) import Path import Path.IO hiding (findExecutable) import Prelude -- Fix AMP warning @@ -255,6 +258,17 @@ (toLazyByteString stdoutBuilder) (toLazyByteString stderrBuilder)) return sinkRet + +logProcessStderrStdout + :: (MonadIO m, MonadBaseControl IO m, MonadLogger m) + => Maybe (Path Abs Dir) + -> String + -> EnvOverride + -> [String] + -> m () +logProcessStderrStdout mdir name menv args = liftBaseWith $ \restore -> do + let logLines = CB.lines =$ CL.mapM_ (void . restore . monadLoggerLog $(TH.location >>= liftLoc) "" LevelInfo . toLogStr) + void $ restore $ sinkProcessStderrStdout mdir menv name args logLines logLines -- | Consume the stdout and stderr of a process feeding strict 'S.ByteString's to the consumers. sinkProcessStderrStdout :: forall m e o. (MonadIO m, MonadLogger m) diff --git a/src/System/Process/Run.hs b/src/System/Process/Run.hs --- a/src/System/Process/Run.hs +++ b/src/System/Process/Run.hs @@ -13,25 +13,30 @@ ,runCmd' ,callProcess ,callProcess' + ,callProcessInheritStderrStdout + ,createProcess' ,ProcessExitedUnsuccessfully ,Cmd(..) ) where import Control.Exception.Lifted -import Control.Monad.Trans.Control (MonadBaseControl) +import Control.Monad (liftM) import Control.Monad.IO.Class (MonadIO, liftIO) import Control.Monad.Logger (MonadLogger, logError) +import Control.Monad.Trans.Control (MonadBaseControl) import Data.Conduit.Process hiding (callProcess) import Data.Foldable (forM_) import Data.Text (Text) import qualified Data.Text as T +import Path (Dir, Abs, Path) import Path (toFilePath) import Prelude -- Fix AMP warning import System.Exit (exitWith, ExitCode (..)) +import System.IO import qualified System.Process +import System.Process.Log import System.Process.Read -import Path (Dir, Abs, Path) -- | Cmd holds common infos needed to running a process in most cases data Cmd = Cmd @@ -92,15 +97,37 @@ -- Inherits stdout and stderr. callProcess' :: (MonadIO m, MonadLogger m) => (CreateProcess -> CreateProcess) -> Cmd -> m () -callProcess' modCP (Cmd wd cmd0 menv args) = do +callProcess' modCP cmd = do + c <- liftM modCP (cmdToCreateProcess cmd) + $logCreateProcess c + liftIO $ do + (_, _, _, p) <- System.Process.createProcess c + exit_code <- waitForProcess p + case exit_code of + ExitSuccess -> return () + ExitFailure _ -> throwIO (ProcessExitedUnsuccessfully c exit_code) + +callProcessInheritStderrStdout :: (MonadIO m, MonadLogger m) => Cmd -> m () +callProcessInheritStderrStdout cmd = do + let inheritOutput cp = cp { std_in = CreatePipe, std_out = Inherit, std_err = Inherit } + callProcess' inheritOutput cmd + +-- | Like 'System.Process.Internal.createProcess_', but taking a 'Cmd'. +-- Note that the 'Handle's provided by 'UseHandle' are not closed +-- automatically. +createProcess' :: (MonadIO m, MonadLogger m) + => String + -> (CreateProcess -> CreateProcess) + -> Cmd + -> m (Maybe Handle, Maybe Handle, Maybe Handle, ProcessHandle) +createProcess' tag modCP cmd = do + c <- liftM modCP (cmdToCreateProcess cmd) + $logCreateProcess c + liftIO $ System.Process.createProcess_ tag c + +cmdToCreateProcess :: MonadIO m => Cmd -> m CreateProcess +cmdToCreateProcess (Cmd wd cmd0 menv args) = do cmd <- preProcess wd menv cmd0 - let c = modCP $ (proc cmd args) { delegate_ctlc = True - , cwd = fmap toFilePath wd - , env = envHelper menv } - action (_, _, _, p) = do - exit_code <- waitForProcess p - case exit_code of - ExitSuccess -> return () - ExitFailure _ -> throwIO (ProcessExitedUnsuccessfully c exit_code) - $logProcessRun cmd args - liftIO (System.Process.createProcess c >>= action) + return $ (proc cmd args) { delegate_ctlc = True + , cwd = fmap toFilePath wd + , env = envHelper menv } diff --git a/src/main/Main.hs b/src/main/Main.hs --- a/src/main/Main.hs +++ b/src/main/Main.hs @@ -16,7 +16,7 @@ import Control.Monad hiding (mapM, forM) import Control.Monad.IO.Class import Control.Monad.Logger -import Control.Monad.Reader (ask, asks, runReaderT) +import Control.Monad.Reader (ask, asks,local,runReaderT) import Control.Monad.Trans.Control (MonadBaseControl) import Data.Attoparsec.Args (parseArgs, EscapingMode (Escaping)) import Data.Attoparsec.Interpreter (getInterpreterArgs) @@ -41,6 +41,7 @@ import Distribution.System (buildArch, buildPlatform) import Distribution.Text (display) import GHC.IO.Encoding (mkTextEncoding, textEncodingName) +import Lens.Micro import Network.HTTP.Client import Options.Applicative import Options.Applicative.Args @@ -115,10 +116,14 @@ , [" (" ++ commitCount ++ " commits)" | commitCount /= ("1"::String) && commitCount /= ("UNKNOWN" :: String)] , [" ", display buildArch] + , [" hpack-", VERSION_hpack] ] where commitCount = $gitCommitCount #else -versionString' = showVersion Meta.version ++ ' ' : display buildArch +versionString' = + showVersion Meta.version + ++ ' ' : display buildArch + ++ " hpack" ++ VERSION_hpack #endif main :: IO () @@ -176,6 +181,7 @@ commandLineHandler progName isInterpreter = complicatedOptions Meta.version (Just versionString') + VERSION_hpack "stack - The Haskell Tool Stack" "" (globalOpts OuterGlobalOpts) @@ -202,30 +208,30 @@ addCommands = do when (not isInterpreter) (do - addCommand' "build" - "Build the package(s) in this directory/configuration" - buildCmd - (buildOptsParser Build) - addCommand' "install" - "Shortcut for 'build --copy-bins'" - buildCmd - (buildOptsParser Install) + addBuildCommand' "build" + "Build the package(s) in this directory/configuration" + buildCmd + (buildOptsParser Build) + addBuildCommand' "install" + "Shortcut for 'build --copy-bins'" + buildCmd + (buildOptsParser Install) addCommand' "uninstall" "DEPRECATED: This command performs no actions, and is present for documentation only" uninstallCmd (many $ strArgument $ metavar "IGNORED") - addCommand' "test" - "Shortcut for 'build --test'" - buildCmd - (buildOptsParser Test) - addCommand' "bench" - "Shortcut for 'build --bench'" - buildCmd - (buildOptsParser Bench) - addCommand' "haddock" - "Shortcut for 'build --haddock'" - buildCmd - (buildOptsParser Haddock) + addBuildCommand' "test" + "Shortcut for 'build --test'" + buildCmd + (buildOptsParser Test) + addBuildCommand' "bench" + "Shortcut for 'build --bench'" + buildCmd + (buildOptsParser Bench) + addBuildCommand' "haddock" + "Shortcut for 'build --haddock'" + buildCmd + (buildOptsParser Haddock) addCommand' "new" "Create a new project from a template. Run `stack templates' to see available templates." newCmd @@ -235,11 +241,11 @@ templatesCmd (pure ()) addCommand' "init" - "Initialize a stack project based on one or more cabal packages" + "Create stack project config from cabal or hpack package specifications" initCmd initOptsParser addCommand' "solver" - "Use a dependency solver to try and determine missing extra-deps" + "Add missing extra-deps to stack project config" solverCmd solverOptsParser addCommand' "setup" @@ -275,23 +281,30 @@ <> help "Clone from specified git repository" <> value "https://github.com/commercialhaskell/stack" <> showDefault )) - addCommand' "upload" - "Upload a package to Hackage" - uploadCmd - ((,,,) - <$> many (strArgument $ metavar "TARBALL/DIR") - <*> optional pvpBoundsOption - <*> ignoreCheckSwitch - <*> flag False True - (long "sign" <> - help "GPG sign & submit signature")) - addCommand' "sdist" - "Create source distribution tarballs" - sdistCmd - ((,,) - <$> many (strArgument $ metavar "DIR") - <*> optional pvpBoundsOption - <*> ignoreCheckSwitch) + addCommand' + "upload" + "Upload a package to Hackage" + uploadCmd + ((,,,,) <$> many (strArgument $ metavar "TARBALL/DIR") <*> + optional pvpBoundsOption <*> + ignoreCheckSwitch <*> + switch (long "no-signature" <> help "Do not sign & upload signatures") <*> + strOption + (long "sig-server" <> metavar "URL" <> showDefault <> + value "https://sig.commercialhaskell.org" <> + help "URL")) + addCommand' + "sdist" + "Create source distribution tarballs" + sdistCmd + ((,,,,) <$> many (strArgument $ metavar "DIR") <*> + optional pvpBoundsOption <*> + ignoreCheckSwitch <*> + switch (long "sign" <> help "Sign & upload signatures") <*> + strOption + (long "sig-server" <> metavar "URL" <> showDefault <> + value "https://sig.commercialhaskell.org" <> + help "URL")) addCommand' "dot" "Visualize your project's dependency graph using Graphviz dot" dotCmd @@ -396,15 +409,22 @@ cfgSetCmd configCmdSetParser) addSubCommands' - Image.imgCmdName - "Subcommands specific to imaging (EXPERIMENTAL)" - (addCommand' Image.imgDockerCmdName - "Build a Docker image for the project" - imgDockerCmd - (boolFlags True - "build" - "building the project before creating the container" - idm)) + Image.imgCmdName + "Subcommands specific to imaging" + (addCommand' + Image.imgDockerCmdName + "Build a Docker image for the project" + imgDockerCmd + ((,) <$> + boolFlags + True + "build" + "building the project before creating the container" + idm <*> + many + (textOption + (long "image" <> + help "A specific container image name to build")))) addSubCommands' "hpc" "Subcommands specific to Haskell Program Coverage" @@ -412,17 +432,6 @@ "Generate HPC report a combined HPC report" hpcReportCmd hpcReportOptsParser) - addSubCommands' - Sig.sigCmdName - "Subcommands specific to package signatures (EXPERIMENTAL)" - (addSubCommands' - Sig.sigSignCmdName - "Sign a a single package or all your packages" - (addCommand' - Sig.sigSignSdistCmdName - "Sign a single sdist package file" - sigSignSdistCmd - Sig.sigSignSdistOpts)) ) where ignoreCheckSwitch = @@ -436,6 +445,10 @@ addSubCommands' cmd title = addSubCommands cmd title globalFooter (globalOpts OtherCmdGlobalOpts) + -- Additional helper that hides global options and shows build options + addBuildCommand' cmd title constr = + addCommand cmd title globalFooter constr (globalOpts BuildCmdGlobalOpts) + globalOpts kind = extraHelpOption hide progName (Docker.dockerCmdName ++ "*") Docker.dockerHelpOptName <*> extraHelpOption hide progName (Nix.nixCmdName ++ "*") Nix.nixHelpOptName <*> @@ -455,7 +468,7 @@ -> IO (ParserFailure ParserHelp) secondaryCommandHandler args f = -- don't even try when the argument looks like a path or flag - if elem pathSeparator cmd || "-" `isPrefixOf` cmd + if elem pathSeparator cmd || "-" `isPrefixOf` (head args) then return f else do mExternalExec <- D.findExecutable cmd @@ -530,19 +543,27 @@ -- So it's not the *minimal* override path. menv <- getMinimalEnvOverride snap <- packageDatabaseDeps - local <- packageDatabaseLocal + plocal <- packageDatabaseLocal extra <- packageDatabaseExtra global <- getGlobalDB menv =<< getWhichCompiler snaproot <- installationRootDeps localroot <- installationRootLocal distDir <- distRelativeDir hpcDir <- hpcReportDir + compiler <- getCompilerPath =<< getWhichCompiler + let deprecated = filter ((`elem` keys) . fst) deprecatedPathKeys + liftIO $ forM_ deprecated $ \(oldOption, newOption) -> T.hPutStrLn stderr $ T.unlines + [ "" + , "'--" <> oldOption <> "' will be removed in a future release." + , "Please use '--" <> newOption <> "' instead." + , "" + ] forM_ -- filter the chosen paths in flags (keys), -- or show all of them if no specific paths chosen. (filter (\(_,key,_) -> - null keys || elem key keys) + (null keys && key /= T.pack deprecatedStackRootOptionName) || elem key keys) paths) (\(_,key,path) -> liftIO $ T.putStrLn @@ -556,32 +577,34 @@ bc menv snap - local + plocal global snaproot localroot distDir hpcDir - extra)))) + extra + compiler)))) -- | Passed to all the path printers as a source of info. data PathInfo = PathInfo - { piBuildConfig :: BuildConfig - , piEnvOverride :: EnvOverride - , piSnapDb :: Path Abs Dir - , piLocalDb :: Path Abs Dir - , piGlobalDb :: Path Abs Dir - , piSnapRoot :: Path Abs Dir - , piLocalRoot :: Path Abs Dir - , piDistDir :: Path Rel Dir - , piHpcDir :: Path Abs Dir - , piExtraDbs :: [Path Abs Dir] + { piBuildConfig :: BuildConfig + , piEnvOverride :: EnvOverride + , piSnapDb :: Path Abs Dir + , piLocalDb :: Path Abs Dir + , piGlobalDb :: Path Abs Dir + , piSnapRoot :: Path Abs Dir + , piLocalRoot :: Path Abs Dir + , piDistDir :: Path Rel Dir + , piHpcDir :: Path Abs Dir + , piExtraDbs :: [Path Abs Dir] + , piCompiler :: Path Abs File } -- | The paths of interest to a user. The first tuple string is used -- for a description that the optparse flag uses, and the second -- string as a machine-readable key and also for @--foo@ flags. The user --- can choose a specific path to list like @--global-stack-root@. But +-- can choose a specific path to list like @--stack-root@. But -- really it's mainly for the documentation aspect. -- -- When printing output we generate @PathInfo@ and pass it to the @@ -590,7 +613,7 @@ paths :: [(String, Text, PathInfo -> Text)] paths = [ ( "Global stack root directory" - , "global-stack-root" + , T.pack stackRootOptionName , T.pack . toFilePathNoTrailingSep . configStackRoot . bcConfig . piBuildConfig ) , ( "Project root (derived from stack.yaml file)" , "project-root" @@ -601,11 +624,17 @@ , ( "PATH environment variable" , "bin-path" , T.pack . intercalate [searchPathSeparator] . eoPath . piEnvOverride ) - , ( "Installed GHCs (unpacked and archives)" - , "ghc-paths" + , ( "Install location for GHC and other core tools" + , "programs" , T.pack . toFilePathNoTrailingSep . configLocalPrograms . bcConfig . piBuildConfig ) - , ( "Local bin path where stack installs executables" - , "local-bin-path" + , ( "Compiler binary (e.g. ghc)" + , "compiler" + , T.pack . toFilePath . piCompiler ) + , ( "Directory containing the compiler binary (e.g. ghc)" + , "compiler-bin" + , T.pack . toFilePathNoTrailingSep . parent . piCompiler ) + , ( "Local bin dir where stack installs executables (e.g. ~/.local/bin)" + , "local-bin" , T.pack . toFilePathNoTrailingSep . configLocalBin . bcConfig . piBuildConfig ) , ( "Extra include directories" , "extra-include-dirs" @@ -642,8 +671,25 @@ , T.pack . toFilePathNoTrailingSep . piDistDir ) , ( "Where HPC reports and tix files are stored" , "local-hpc-root" - , T.pack . toFilePathNoTrailingSep . piHpcDir ) ] + , T.pack . toFilePathNoTrailingSep . piHpcDir ) + , ( "DEPRECATED: Use '--local-bin' instead" + , "local-bin-path" + , T.pack . toFilePathNoTrailingSep . configLocalBin . bcConfig . piBuildConfig ) + , ( "DEPRECATED: Use '--programs' instead" + , "ghc-paths" + , T.pack . toFilePathNoTrailingSep . configLocalPrograms . bcConfig . piBuildConfig ) + , ( "DEPRECATED: Use '--" <> stackRootOptionName <> "' instead" + , T.pack deprecatedStackRootOptionName + , T.pack . toFilePathNoTrailingSep . configStackRoot . bcConfig . piBuildConfig ) + ] +deprecatedPathKeys :: [(Text, Text)] +deprecatedPathKeys = + [ (T.pack deprecatedStackRootOptionName, T.pack stackRootOptionName) + , ("ghc-paths", "programs") + , ("local-bin-path", "local-bin") + ] + data SetupCmdOpts = SetupCmdOpts { scoCompilerVersion :: !(Maybe CompilerVersion) , scoForceReinstall :: !Bool @@ -812,7 +858,7 @@ withBuildConfigExt :: GlobalOpts -> Maybe (StackT Config IO ()) - -- ^ Action to perform after before build. This will be run on the host + -- ^ Action to perform before the build. This will be run on the host -- OS even if Docker is enabled for builds. The build config is not -- available in this action, since that would require build tools to be -- installed on the host OS. @@ -870,20 +916,27 @@ cleanCmd opts go = withBuildConfigAndLock go (const (clean opts)) -- | Helper for build and install commands -buildCmd :: BuildOpts -> GlobalOpts -> IO () +buildCmd :: BuildOptsCLI -> GlobalOpts -> IO () buildCmd opts go = do - when (any (("-prof" `elem`) . either (const []) id . parseArgs Escaping) (boptsGhcOptions opts)) $ do + when (any (("-prof" `elem`) . either (const []) id . parseArgs Escaping) (boptsCLIGhcOptions opts)) $ do hPutStrLn stderr "When building with stack, you should not use the -prof GHC option" hPutStrLn stderr "Instead, please use --library-profiling and --executable-profiling" hPutStrLn stderr "See: https://github.com/commercialhaskell/stack/issues/1015" error "-prof GHC option submitted" - case boptsFileWatch opts of + case boptsCLIFileWatch opts of FileWatchPoll -> fileWatchPoll stderr inner FileWatch -> fileWatch stderr inner NoFileWatch -> inner $ const $ return () where - inner setLocalFiles = withBuildConfigAndLock go $ \lk -> + inner setLocalFiles = withBuildConfigAndLock go' $ \lk -> Stack.Build.build setLocalFiles lk opts + -- Read the build command from the CLI and enable it to run + go' = case boptsCLICommand opts of + Test -> set (globalOptsBuildOptsMonoid.buildOptsMonoidTests) (Just True) go + Haddock -> set (globalOptsBuildOptsMonoid.buildOptsMonoidHaddock) (Just True) go + Bench -> set (globalOptsBuildOptsMonoid.buildOptsMonoidBenchmarks) (Just True) go + Install -> set (globalOptsBuildOptsMonoid.buildOptsMonoidInstallExes) (Just True) go + Build -> go -- Default case is just Build uninstallCmd :: [String] -> GlobalOpts -> IO () uninstallCmd _ go = withConfigAndLock go $ do @@ -913,9 +966,9 @@ #endif -- | Upload to Hackage -uploadCmd :: ([String], Maybe PvpBounds, Bool, Bool) -> GlobalOpts -> IO () -uploadCmd ([], _, _, _) _ = error "To upload the current package, please run 'stack upload .'" -uploadCmd (args, mpvpBounds, ignoreCheck, shouldSign) go = do +uploadCmd :: ([String], Maybe PvpBounds, Bool, Bool, String) -> GlobalOpts -> IO () +uploadCmd ([], _, _, _, _) _ = error "To upload the current package, please run 'stack upload .'" +uploadCmd (args, mpvpBounds, ignoreCheck, don'tSign, sigServerUrl) go = do let partitionM _ [] = return ([], []) partitionM f (x:xs) = do r <- f x @@ -926,7 +979,6 @@ unless (null invalid) $ error $ "stack upload expects a list sdist tarballs or cabal directories. Can't find " ++ show invalid - (_,lc) <- liftIO $ loadConfigWithOpts go let getUploader :: (HasStackRoot config, HasPlatform config, HasConfig config) => StackT config IO Upload.Uploader getUploader = do config <- asks getConfig @@ -934,9 +986,9 @@ let uploadSettings = Upload.setGetManager (return manager) Upload.defaultUploadSettings liftIO $ Upload.mkUploader config uploadSettings - sigServiceUrl = "https://sig.commercialhaskell.org/" withBuildConfigAndLock go $ \_ -> do uploader <- getUploader + manager <- asks envManager unless ignoreCheck $ mapM_ (resolveFile' >=> checkSDistTarball) files forM_ @@ -945,11 +997,12 @@ do tarFile <- resolveFile' file liftIO (Upload.upload uploader (toFilePath tarFile)) - when - shouldSign - (Sig.sign - (lcProjectRoot lc) - sigServiceUrl + unless + don'tSign + (void $ + Sig.sign + manager + sigServerUrl tarFile)) unless (null dirs) $ forM_ dirs $ \dir -> do @@ -958,21 +1011,23 @@ unless ignoreCheck $ checkSDistTarball' tarName tarBytes liftIO $ Upload.uploadBytes uploader tarName tarBytes tarPath <- parseRelFile tarName - when - shouldSign - (Sig.signTarBytes - (lcProjectRoot lc) - sigServiceUrl + unless + don'tSign + (void $ + Sig.signTarBytes + manager + sigServerUrl tarPath tarBytes) -sdistCmd :: ([String], Maybe PvpBounds, Bool) -> GlobalOpts -> IO () -sdistCmd (dirs, mpvpBounds, ignoreCheck) go = +sdistCmd :: ([String], Maybe PvpBounds, Bool, Bool, String) -> GlobalOpts -> IO () +sdistCmd (dirs, mpvpBounds, ignoreCheck, sign, sigServerUrl) go = withBuildConfig go $ do -- No locking needed. -- If no directories are specified, build all sdist tarballs. dirs' <- if null dirs then asks (Map.keys . envConfigPackages . getEnvConfig) else mapM resolveDir' dirs + manager <- asks envManager forM_ dirs' $ \dir -> do (tarName, tarBytes) <- getSDistTarball mpvpBounds dir distDir <- distDirFromDir dir @@ -981,6 +1036,7 @@ liftIO $ L.writeFile (toFilePath tarPath) tarBytes unless ignoreCheck (checkSDistTarball tarPath) $logInfo $ "Wrote sdist tarball to " <> T.pack (toFilePath tarPath) + when sign (void $ Sig.sign manager sigServerUrl tarPath) -- | Execute a command. execCmd :: ExecOpts -> GlobalOpts -> IO () @@ -1020,8 +1076,8 @@ (ExecRunGhc, args) -> execCompiler "run" args let targets = concatMap words eoPackages unless (null targets) $ - Stack.Build.build (const $ return ()) lk defaultBuildOpts - { boptsTargets = map T.pack targets + Stack.Build.build (const $ return ()) lk defaultBuildOptsCLI + { boptsCLITargets = map T.pack targets } munlockFile lk -- Unlock before transferring control away. menv <- liftIO $ configEnvOverride config eoEnvSettings @@ -1047,7 +1103,14 @@ ghciCmd ghciOpts go@GlobalOpts{..} = withBuildConfigAndLock go $ \lk -> do munlockFile lk -- Don't hold the lock while in the GHCI. - ghci ghciOpts + bopts <- asks (configBuild . getConfig) + -- override env so running of tests and benchmarks is disabled + let boptsLocal = bopts + { boptsTestOpts = (boptsTestOpts bopts) { toDisableRun = True } + , boptsBenchmarkOpts = (boptsBenchmarkOpts bopts) { beoDisableRun = True } + } + local (set (envEnvConfig.envConfigBuildOpts) boptsLocal) + (ghci ghciOpts) -- | Run ide-backend in the context of a project. ideCmd :: ([Text], [String]) -> GlobalOpts -> IO () @@ -1071,8 +1134,8 @@ targetsCmd :: Text -> GlobalOpts -> IO () targetsCmd target go@GlobalOpts{..} = withBuildConfig go $ - do let bopts = defaultBuildOpts { boptsTargets = [target] } - (_realTargets,_,pkgs) <- ghciSetup (ideGhciOpts bopts) + do let boptsCli = defaultBuildOptsCLI { boptsCLITargets = [target] } + (_realTargets,_,pkgs) <- ghciSetup (ideGhciOpts boptsCli) pwd <- getCurrentDir targets <- fmap @@ -1117,30 +1180,20 @@ (cfgCmdSet co) env) -imgDockerCmd :: Bool -> GlobalOpts -> IO () -imgDockerCmd rebuild go@GlobalOpts{..} = +imgDockerCmd :: (Bool, [Text]) -> GlobalOpts -> IO () +imgDockerCmd (rebuild,images) go@GlobalOpts{..} = do + mProjectRoot <- lcProjectRoot . snd <$> loadConfigWithOpts go withBuildConfigExt go Nothing (\lk -> - do when rebuild $ Stack.Build.build + do when rebuild $ + Stack.Build.build (const (return ())) lk - defaultBuildOpts - Image.stageContainerImageArtifacts) - (Just Image.createContainerImageFromStage) - -sigSignSdistCmd :: (String, String) -> GlobalOpts -> IO () -sigSignSdistCmd (url,path) go = - withConfigAndLock - go - (do (manager,lc) <- liftIO (loadConfigWithOpts go) - tarBall <- resolveFile' path - runStackTGlobal - manager - (lcConfig lc) - go - (Sig.sign (lcProjectRoot lc) url tarBall)) + defaultBuildOptsCLI + Image.stageContainerImageArtifacts mProjectRoot) + (Just $ Image.createContainerImageFromStage mProjectRoot images) -- | Load the configuration with a manager. Convenience function used -- throughout this module. diff --git a/src/test/Stack/ArgsSpec.hs b/src/test/Stack/ArgsSpec.hs --- a/src/test/Stack/ArgsSpec.hs +++ b/src/test/Stack/ArgsSpec.hs @@ -46,6 +46,10 @@ checkLines "" checkLines " --x" checkLines " --x --y" + describe "Literate line comments" $ do + checkLiterateLines "" + checkLiterateLines " --x" + checkLiterateLines " --x --y" describe "Block comments" $ do checkBlocks "" checkBlocks "\n" @@ -54,16 +58,31 @@ checkBlocks " --x --y" checkBlocks "\n--x\n--y" checkBlocks "\n\t--x\n\t--y" + describe "Literate block comments" $ do + checkLiterateBlocks "" "" + checkLiterateBlocks "\n>" "" + checkLiterateBlocks " --x" " --x" + checkLiterateBlocks "\n>--x" "--x" + checkLiterateBlocks " --x --y " "--x --y" + checkLiterateBlocks "\n>--x\n>--y" "--x --y" + checkLiterateBlocks "\n>\t--x\n>\t--y" "--x --y" describe "Failure cases" $ do checkFailures + describe "Bare directives in literate files" $ do + forM_ (interpreterGenValid lineComment []) $ + testAndCheck (acceptFailure True) [] + forM_ (interpreterGenValid blockComment []) $ + testAndCheck (acceptFailure True) [] where - parse s = P.parseOnly (interpreterArgsParser stackProgName) (pack s) + parse isLiterate s = + P.parseOnly (interpreterArgsParser isLiterate stackProgName) (pack s) - acceptSuccess args s = case parse s of + acceptSuccess :: Bool -> String -> String -> Bool + acceptSuccess isLiterate args s = case parse isLiterate s of Right x | words x == words args -> True _ -> False - acceptFailure _ s = case parse s of + acceptFailure isLiterate _ s = case parse isLiterate s of Left _ -> True Right _ -> False @@ -72,19 +91,28 @@ checkLines args = forM_ (interpreterGenValid lineComment args) - (testAndCheck acceptSuccess args) + (testAndCheck (acceptSuccess False) args) + checkLiterateLines args = forM_ + (interpreterGenValid literateLineComment args) + (testAndCheck (acceptSuccess True) args) + checkBlocks args = forM_ (interpreterGenValid blockComment args) - (testAndCheck acceptSuccess args) + (testAndCheck (acceptSuccess False) args) + checkLiterateBlocks inp args = forM_ + (interpreterGenValid literateBlockComment inp) + (testAndCheck (acceptSuccess True) args) + checkFailures = forM_ interpreterGenInvalid - (testAndCheck acceptFailure "unused") + (testAndCheck (acceptFailure False) "unused") -- Generate a set of acceptable inputs for given format and args interpreterGenValid fmt args = shebang <++> newLine <++> (fmt args) + interpreterGenInvalid :: [String] -- Generate a set of Invalid inputs interpreterGenInvalid = ["-stack\n"] -- random input @@ -128,6 +156,12 @@ lineComment = makeComment makeLine lineSpace where makeLine s = "--" ++ s + literateLineComment = makeComment ("> --" ++) lineSpace + blockSpace = lineSpace <|> newLine blockComment = makeComment makeBlock blockSpace where makeBlock s = "{-" ++ s ++ "-}" + + literateBlockComment = makeComment + (\s -> "> {-" ++ s ++ "-}") + (lineSpace <|> map (++ ">") newLine) diff --git a/src/test/Stack/ConfigSpec.hs b/src/test/Stack/ConfigSpec.hs --- a/src/test/Stack/ConfigSpec.hs +++ b/src/test/Stack/ConfigSpec.hs @@ -27,6 +27,32 @@ "resolver: lts-2.10\n" ++ "packages: ['.']\n" +buildOptsConfig :: String +buildOptsConfig = + "resolver: lts-2.10\n" ++ + "packages: ['.']\n" ++ + "build:\n" ++ + " library-profiling: true\n" ++ + " executable-profiling: true\n" ++ + " haddock: true\n" ++ + " haddock-deps: true\n" ++ + " copy-bins: true\n" ++ + " prefetch: true\n" ++ + " force-dirty: true\n" ++ + " keep-going: true\n" ++ + " test: true\n" ++ + " test-arguments:\n" ++ + " rerun-tests: true\n" ++ + " additional-args: ['-fprof']\n" ++ + " coverage: true\n" ++ + " no-run-tests: true\n" ++ + " bench: true\n" ++ + " benchmark-opts:\n" ++ + " benchmark-arguments: -O2\n" ++ + " no-run-benchmarks: true\n" ++ + " reconfigure: true\n" ++ + " cabal-verbose: true\n" + stackDotYaml :: Path Rel File stackDotYaml = $(mkRelFile "stack.yaml") @@ -43,6 +69,9 @@ teardown :: T -> IO () teardown _ = return () +noException :: Selector SomeException +noException = const False + spec :: Spec spec = beforeAll setup $ afterAll teardown $ do let logLevel = LevelDebug @@ -72,6 +101,28 @@ writeFile (toFilePath stackDotYaml) "" -- TODO(danburton): more specific test for exception loadConfig' manager `shouldThrow` anyException + + it "parses build config options" $ \T{..} -> inTempDir $ do + writeFile (toFilePath stackDotYaml) buildOptsConfig + BuildOpts{..} <- configBuild . lcConfig <$> loadConfig' manager + boptsLibProfile `shouldBe` True + boptsExeProfile `shouldBe` True + boptsHaddock `shouldBe` True + boptsHaddockDeps `shouldBe` (Just True) + boptsInstallExes `shouldBe` True + boptsPreFetch `shouldBe` True + boptsKeepGoing `shouldBe` (Just True) + boptsForceDirty `shouldBe` True + boptsTests `shouldBe` True + boptsTestOpts `shouldBe` (TestOpts {toRerunTests = True + ,toAdditionalArgs = ["-fprof"] + ,toCoverage = True + ,toDisableRun = True}) + boptsBenchmarks `shouldBe` True + boptsBenchmarkOpts `shouldBe` (BenchmarkOpts {beoAdditionalArgs = Just "-O2" + ,beoDisableRun = True}) + boptsReconfigure `shouldBe` True + boptsCabalVerbose `shouldBe` True it "finds the config file in a parent directory" $ \T{..} -> inTempDir $ do writeFile (toFilePath stackDotYaml) sampleConfig diff --git a/src/test/Stack/NixSpec.hs b/src/test/Stack/NixSpec.hs --- a/src/test/Stack/NixSpec.hs +++ b/src/test/Stack/NixSpec.hs @@ -3,24 +3,21 @@ {-# LANGUAGE TemplateHaskell, OverloadedStrings #-} module Stack.NixSpec where -import Test.Hspec - -import Control.Monad.Logger import Control.Exception +import Control.Monad.Logger import Data.Monoid import Network.HTTP.Conduit (Manager) -import System.Environment import Path -import System.Directory -import System.IO.Temp (withSystemTempDirectory) - +import Prelude -- to remove the warning about Data.Monoid being redundant on GHC 7.10 import Stack.Config +import Stack.Config.Nix import Stack.Types.Config -import Stack.Types.StackT import Stack.Types.Nix - -import Prelude -- to remove the warning about Data.Monoid being redundant on GHC 7.10 - +import Stack.Types.StackT +import System.Directory +import System.Environment +import System.IO.Temp (withSystemTempDirectory) +import Test.Hspec sampleConfig :: String sampleConfig = @@ -64,4 +61,4 @@ writeFile (toFilePath stackDotYaml) sampleConfig lc <- loadConfig' manager (nixPackages $ configNix $ lcConfig lc) `shouldBe` ["glpk"] - (nixCompiler $ configNix $ lcConfig lc) Nothing Nothing `shouldBe` "haskell.packages.lts-2_10.ghc" + nixCompiler (lcConfig lc) Nothing Nothing `shouldBe` "haskell.packages.lts-2_10.ghc" diff --git a/stack.cabal b/stack.cabal --- a/stack.cabal +++ b/stack.cabal @@ -1,5 +1,5 @@ name: stack -version: 1.0.4.3 +version: 1.1.0 cabal-version: >=1.10 build-type: Simple license: BSD3 @@ -19,6 +19,7 @@ CONTRIBUTING.md ChangeLog.md README.md + doc/*.md test/package-dump/ghc-7.8.txt test/package-dump/ghc-7.8.4-osx.txt test/package-dump/ghc-7.10.txt @@ -64,7 +65,6 @@ Data.Binary.VersionTagged Data.IORef.RunOnce Data.Maybe.Extra - Data.Set.Monad Distribution.Version.Extra Network.HTTP.Download Network.HTTP.Download.Verified @@ -85,6 +85,8 @@ Stack.BuildPlan Stack.Clean Stack.Config + Stack.Config.Build + Stack.Config.Urls Stack.Config.Docker Stack.Config.Nix Stack.ConfigCmd @@ -117,8 +119,10 @@ Stack.Types Stack.Types.Build Stack.Types.BuildPlan + Stack.Types.Urls Stack.Types.Compiler Stack.Types.Config + Stack.Types.Config.Build Stack.Types.Docker Stack.Types.FlagName Stack.Types.GhcPkgId @@ -150,7 +154,7 @@ base64-bytestring >=1.0.0.1 && <1.1, binary ==0.7.*, binary-tagged >=0.1.1 && <0.2, - blaze-builder >=0.4.0.1 && <0.5, + blaze-builder >=0.4.0.2 && <0.5, byteable >=0.1.1 && <0.2, bytestring >=0.10.6.0 && <0.11, conduit >=1.2.4 && <1.3, @@ -162,7 +166,7 @@ edit-distance ==0.2.*, either >=4.4.1 && <4.5, enclosed-exceptions >=1.0.1.1 && <1.1, - errors >=2.1.1 && <2.2, + errors >=2.1.2 && <2.2, exceptions >=0.8.0.2 && <0.9, extra >=1.4.3 && <1.5, fast-logger >=2.3.1 && <2.5, @@ -176,15 +180,17 @@ http-conduit >=2.1.7 && <2.2, http-types >=0.8.6 && <0.10, lifted-base >=0.2.3.6 && <0.3, - monad-control >=1.0.0.4 && <1.1, + microlens >=0.3.0.0 && <0.5, + monad-control >=1.0.1.0 && <1.1, monad-logger >=0.3.13.1 && <0.4, mtl >=2.1.3.1 && <2.3, + open-browser >=0.2.1 && <0.3, optparse-applicative >=0.11 && <0.13, path >=0.5.1 && <0.6, - path-io >=1.0.0 && <2.0.0, - persistent >=2.1.2 && <2.3, - persistent-sqlite >=2.1.4 && <2.3, - persistent-template >=2.1.1 && <2.2, + path-io >=1.1.0 && <2.0.0, + persistent >=2.1.2 && <2.6, + persistent-sqlite >=2.1.4 && <2.6, + persistent-template >=2.1.1 && <2.6, pretty >=1.1.1.1 && <1.2, process >=1.2.0.0 && <1.3, resourcet >=1.1.4.1 && <1.2, @@ -205,15 +211,14 @@ unix-compat >=0.4.1.4 && <0.5, unordered-containers >=0.2.5.1 && <0.3, vector >=0.10.12.3 && <0.12, - vector-binary-instances >=0.2.1.0 && <0.3, + vector-binary-instances >=0.2.3.2 && <0.3, yaml >=0.8.10.1 && <0.9, zlib >=0.5.4.2 && <0.7, deepseq ==1.4.*, hastache >=0.6.1 && <0.7, project-template ==0.2.*, - uuid >=1.3.11 && <1.4, zip-archive >=0.2.3.7 && <0.3, - hpack >=0.9.0 && <0.12 + hpack >=0.10.0 && <0.14 default-language: Haskell2010 hs-source-dirs: src/ ghc-options: -Wall -fwarn-tabs -fwarn-incomplete-uni-patterns -fwarn-incomplete-record-updates @@ -242,22 +247,23 @@ directory >=1.2.2.0 && <1.3, filelock >=0.1.0.1 && <0.2, filepath >=1.4.0.0 && <1.5, - http-client >=0.4.27 && <0.5, + http-client >=0.4.28 && <0.5, lifted-base >=0.2.3.6 && <0.3, - monad-control >=1.0.0.4 && <1.1, + microlens >=0.3.0.0 && <0.5, + monad-control >=1.0.1.0 && <1.1, monad-logger >=0.3.13.1 && <0.4, mtl >=2.1.3.1 && <2.3, optparse-applicative >=0.11.0.2 && <0.13, - path >=0.5.3 && <0.6, - path-io >=1.0.0 && <2.0.0, - stack >=1.0.4.3 && <1.1, + path >=0.5.7 && <0.6, + path-io >=1.1.0 && <2.0.0, + stack >=1.1.0 && <1.2, text >=1.2.0.4 && <1.3, transformers >=0.4.2.0 && <0.5 default-language: Haskell2010 hs-source-dirs: src/main other-modules: Paths_stack - ghc-options: -threaded -rtsopts -Wall -fwarn-tabs -fwarn-incomplete-uni-patterns -fwarn-incomplete-record-updates + ghc-options: -threaded -Wall -fwarn-tabs -fwarn-incomplete-uni-patterns -fwarn-incomplete-record-updates test-suite stack-test type: exitcode-stdio-1.0 @@ -265,24 +271,24 @@ build-depends: Cabal >=1.18.1.5 && <1.23, QuickCheck >=2.8.1 && <2.9, - attoparsec >=0.13.0.1 && <0.14, + attoparsec >=0.13.0.2 && <0.14, base >=4.7 && <5, - conduit >=1.2.6.1 && <1.3, - conduit-extra >=1.1.9.2 && <1.2, + conduit >=1.2.6.4 && <1.3, + conduit-extra >=1.1.13.1 && <1.2, containers >=0.5.5.1 && <0.6, cryptohash >=0.11.6 && <0.12, directory >=1.2.2.0 && <1.3, exceptions >=0.8.2.1 && <0.9, - hspec >=2.2.2 && <2.3, - http-conduit >=2.1.8 && <2.2, - monad-logger >=0.3.17 && <0.4, - path >=0.5.3 && <0.6, - path-io >=1.0.0 && <2.0.0, - resourcet >=1.1.7 && <1.2, + hspec >=2.2.3 && <2.3, + http-conduit >=2.1.10 && <2.2, + monad-logger >=0.3.18 && <0.4, + path >=0.5.7 && <0.6, + path-io >=1.1.0 && <2.0.0, + resourcet >=1.1.7.3 && <1.2, retry >=0.6 && <0.8, - stack >=1.0.4.3 && <1.1, + stack >=1.1.0 && <1.2, temporary >=1.2.0.4 && <1.3, - text >=1.2.2.0 && <1.3, + text >=1.2.2.1 && <1.3, transformers >=0.4.2.0 && <0.5 default-language: Haskell2010 hs-source-dirs: src/test @@ -308,16 +314,16 @@ async >=2.1.0 && <2.2, base >=4.7 && <5, bytestring >=0.10.6.0 && <0.11, - conduit >=1.2.6.1 && <1.3, - conduit-extra >=1.1.9.2 && <1.2, + conduit >=1.2.6.4 && <1.3, + conduit-extra >=1.1.13.1 && <1.2, containers >=0.5.5.1 && <0.6, directory >=1.2.2.0 && <1.3, filepath >=1.4.0.0 && <1.5, - hspec >=2.2.2 && <2.3, + hspec >=2.2.3 && <2.3, process >=1.2.3.0 && <1.3, - resourcet >=1.1.7 && <1.2, + resourcet >=1.1.7.3 && <1.2, temporary >=1.2.0.4 && <1.3, - text >=1.2.2.0 && <1.3, + text >=1.2.2.1 && <1.3, transformers >=0.4.2.0 && <0.5, unix-compat >=0.4.1.4 && <0.5 default-language: Haskell2010 diff --git a/stack.yaml b/stack.yaml --- a/stack.yaml +++ b/stack.yaml @@ -1,14 +1,17 @@ -resolver: lts-5.0 -image: - containers: - - base: "fpco/ubuntu-with-libgmp:14.04" - entrypoints: - - stack +resolver: lts-5.14 +# docker: +# enable: true +# repo: fpco/stack-full +# image: +# containers: +# - base: "fpco/stack-base" # see ./etc/docker/stack-base/Dockerfile +# name: "fpco/stack-test" nix: # --nix on the command-line to enable. enable: false packages: - zlib extra-deps: -- path-io-1.0.0 -- hpack-0.9.0 +- hpack-0.13.0 +- path-io-1.1.0 +- th-lift-instances-0.1.6 diff --git a/test/integration/IntegrationSpec.hs b/test/integration/IntegrationSpec.hs --- a/test/integration/IntegrationSpec.hs +++ b/test/integration/IntegrationSpec.hs @@ -71,7 +71,8 @@ -> String -> Spec test runghc env' currDir origStackRoot newHome name = it name $ withDir $ \dir -> do - removeDirectoryRecursive newHome + newHomeExists <- doesDirectoryExist newHome + when newHomeExists (removeDirectoryRecursive newHome) copyTree toCopyRoot origStackRoot (newHome takeFileName origStackRoot) let testDir = currDir "tests" name mainFile = testDir "Main.hs" diff --git a/test/integration/lib/StackTest.hs b/test/integration/lib/StackTest.hs --- a/test/integration/lib/StackTest.hs +++ b/test/integration/lib/StackTest.hs @@ -122,6 +122,9 @@ -- | Is the OS Windows? isWindows = os == "mingw32" +-- | Is the OS Mac OS X? +isMacOSX = os == "darwin" + -- | To avoid problems with GHC version mismatch when a new LTS major -- version is released, pass this argument to @stack@ when running in -- a global context. The LTS major version here should match that of