diff --git a/CHANGELOG.md b/CHANGELOG.md --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,4 @@ # 0.3.0 - - Initial Hackage release - Add CHANGELOG.md @@ -8,3 +7,21 @@ - Add Nix wrapper, see `nix/README.md` - Upd support cabal2nix >2.5 - Fix honor `--hackage-db` flag + +# 0.5.0 +- Add `--with-stackage` and `--with-stackage-closure` flag generates stackage +- Add `--extra-deps-revision-latest` flag. Changes generation strategy for the + direct dependencies of the stack.yaml extra-deps. Default strategy is to + generate exact revision defined in the Stackage config. With this flag + enabled, direct dependencies of the stack.yaml extra-deps will be generated + with latest revision available on Hackage. This may help when Cabal fails to + resolve dependencies of extra-deps, and dependences are fixed in the latest + revision +- Upd by default stackage2nix generates override of pre-generated Stackage + packages set. +- Upd generated Stackage packages with `doCheck` and `runHaddock` enabled by + default +- Remove `--do-check-stackage` flag (enabled by default) +- Remove `--do-haddock-stackage` flag (enabled by default) +- Remove `--no-packages-closure` flag (use `--with-stackage-closure`) +- Remove `cabal2nix < 2.7.2` support #45 diff --git a/README.md b/README.md --- a/README.md +++ b/README.md @@ -3,73 +3,78 @@ [![Build Status](https://travis-ci.org/typeable/stackage2nix.svg?branch=master)](https://travis-ci.org/typeable/stackage2nix) `stackage2nix` converts a Stack file into a Nix Haskell packages set. -It creates LTS Stackage packages set, and applies appropriate overrides on top of it. -## Install +# Create build derivation from stack.yaml -You can install `stackage2nix` from the Nix expression: +## Generate targets from stack.yaml only ``` -nix-env -i -f ./nix/stackage2nix +stack exec -- stackage2nix . ``` -It provides pre-configured wrapper around the raw executable with runtime -`PATH` and all auxiliary flags set up. - -## Build project +Command creates file `default.nix` which overrides `haskell.packages.stackage` +packages set. You should use +[typeable/nixpkgs-stackage](https://github.com/typeable/nixpkgs-stackage) overlay that +adds LTS Stackage packages to Nixpkgs. -Generate derivations from `stack.yaml` config using Nix wrapper: +Build package with overlay installed: -``` bash -stackage2nix ./stack.yaml ``` +nix-build -A stackage2nix +``` -if you're using the raw executable, you should supply additional flags. See -section 'Flags' below for details. +## Generate Stackage packages for the build -This command will result in a Haskell packages set, similar to -`pkgs.haskell.packages.`, containing only packages that are required -to build targets listed in `stackage.yaml`. To build a package run: +If you don't want to use Stackage overlay, stackage2nix can generate required +packages with `--with-stackage-closure` flag. ``` bash -nix-build -A my-package +stack exec -- stackage2nix \ + --all-cabal-hashes /path/to/commercialhaskell/all-cabal-hashes \ + --lts-haskell /path/to/fpco/lts-haskell \ + --with-stackage-closure \ + ./stack.yaml ``` -## Build stackage +To generate Stackage packages, you should supply additional +`--all-cabal-hashes` and `--lts-haskell` flags that points to +[commercialhaskell/all-cabal-hashes](https://github.com/commercialhaskell) +checked out to `hackage` branch and +[fpco/lts-haskell](https://github.com/fpco/lts-haskell) respectively. -Generate complete Stackage packages set from resolver: +### stackage2nix wrapper -``` bash -stackage2nix --resolver lts-9.0 +You can use stackage2nix wrapper from `nix` directory that adds required flags: + ``` +nix-env -i -f ./nix/stackage2nix +stackage2nix --with-stackage-closure ./stack.yaml +``` -## Flags +This command will produce `packages.nix` and `configuration-packages.nix` +Stackage packages and its override in `default.nix` +## Generate full Stackage + +`--with-stackage` parameter generates full Stackage LTS in addition to the +targets from `stack.yaml` + ``` -stackage2nix \ - --lts-haskell "$LTS_HASKELL_REPO" \ - --all-cabal-hashes "$ALL_CABAL_HASHES_REPO" \ - . +nix-env -i -f ./nix/stackage2nix +stackage2nix --with-stackage ./stack.yaml ``` -`stackage2nix` has three required arguments: -- `--lts-haskell` - path to [fpco/lts-haskell](https://github.com/fpco/lts-haskell) -- `--all-cabal-hashes` - path to [commercialhaskell/all-cabal-hashes](https://github.com/commercialhaskell/all-cabal-hashes) checked out to `hackage` branch -- `.` - path to stack.yaml file or directory - -Produced Nix derivation split on following files: -- packages.nix - Base Stackage packages set -- configuration-packages.nix - Compiler configuration -- default.nix - Final Haskell packages set with all overrides applied +## Generate Stackage from LTS resolver -A particular package from result Haskell packages set can be built with: +You can also generate only Stackage packages set from the resolver: ``` -nix-build -A +nix-env -i -f ./nix/stackage2nix +stackage2nix --resolver lts-10.0 ``` -See also the [blog post](https://blog.typeable.io/posts/2017-08-24-stackage2nix.html) -about history and motivation behind the project. +This command will produce Stackage packages `packages.nix`, packages config +`configuration-packages.nix` and a Haskell packages set `default.nix`. ## Runtime dependencies @@ -85,9 +90,9 @@ Snippet `override.nix` below shows a minimal example of how to apply additional overrides on top of Haskell packages set produced by `stackage2nix`. -``` +``` nix with import {}; -with pkgs.haskell.lib; +with haskell.lib; let haskellPackages = import ./. {}; in haskellPackages.override { overrides = self: super: { @@ -104,9 +109,9 @@ ## Tests -Integration tests that build stackage2nix form different stack configs: +Integration tests that build stackage2nix form different yaml configs: -``` +``` bash STACKAGE_REPO= \ ALL_CABAL_HASHES= \ STACK_FILE=stack-ghc-7103.yaml \ diff --git a/src/Distribution/Nixpkgs/Haskell/Stack.hs b/src/Distribution/Nixpkgs/Haskell/Stack.hs --- a/src/Distribution/Nixpkgs/Haskell/Stack.hs +++ b/src/Distribution/Nixpkgs/Haskell/Stack.hs @@ -1,4 +1,3 @@ -{-# LANGUAGE CPP #-} module Distribution.Nixpkgs.Haskell.Stack where import Control.Lens @@ -9,6 +8,7 @@ import Distribution.Nixpkgs.Haskell.Derivation import Distribution.Nixpkgs.Haskell.FromCabal as FromCabal import Distribution.Nixpkgs.Haskell.FromStack as FromStack +import Distribution.Nixpkgs.Haskell.Hackage as DB import Distribution.Nixpkgs.Haskell.PackageSourceSpec as PackageSourceSpec import Distribution.PackageDescription as PackageDescription import Distribution.System as System @@ -16,44 +16,25 @@ import Stack.Config -newtype HackageDb = HackageDb { fromHackageDB :: T.Text } - deriving (Eq, Ord, Show) -makePrisms ''HackageDb - -unHackageDb :: HackageDb -> String -unHackageDb = T.unpack . fromHackageDB - data StackPackagesConfig = StackPackagesConfig - { _spcHaskellResolver :: !HaskellResolver - , _spcNixpkgsResolver :: !NixpkgsResolver - , _spcTargetPlatform :: !Platform - , _spcTargetCompiler :: !CompilerInfo - , _spcFlagAssignment :: !FlagAssignment - , _spcDoCheckPackages :: !Bool - , _spcDoHaddockPackages :: !Bool - , _spcDoCheckStackage :: !Bool - , _spcDoHaddockStackage :: !Bool + { _spcHaskellResolver :: HaskellResolver + , _spcNixpkgsResolver :: NixpkgsResolver + , _spcTargetPlatform :: Platform + , _spcTargetCompiler :: CompilerInfo + , _spcFlagAssignment :: FlagAssignment + , _spcDoCheckPackages :: Bool + , _spcDoHaddockPackages :: Bool } makeLenses ''StackPackagesConfig -getStackPackageFromDb - :: Maybe HackageDb - -> StackPackage - -> IO Package -getStackPackageFromDb optHackageDb stackPackage = -#if MIN_VERSION_cabal2nix(2,6,0) - PackageSourceSpec.getPackage +getStackPackageFromDb :: DB.HackageDB -> StackPackage -> IO Package +getStackPackageFromDb hackageDb stackPackage = do + PackageSourceSpec.getPackage' False - (unHackageDb <$> optHackageDb) - Nothing - (stackLocationToSource (stackPackage ^. spLocation) (stackPackage ^. spDir)) -#else - PackageSourceSpec.getPackage - (unHackageDb <$> optHackageDb) + (pure hackageDb) (stackLocationToSource (stackPackage ^. spLocation) (stackPackage ^. spDir)) -#endif stackLocationToSource :: PackageLocation @@ -86,18 +67,17 @@ packageDerivation :: StackPackagesConfig - -> Maybe HackageDb + -> DB.HackageDB -> StackPackage -> IO Derivation -packageDerivation conf optHackageDb stackPackage = do - pkg <- getStackPackageFromDb optHackageDb stackPackage +packageDerivation conf hackageDb stackPackage = do + pkg <- getStackPackageFromDb hackageDb stackPackage let drv = genericPackageDerivation conf pkg & subpath %~ flip fromMaybe (stackPackage ^. spDir) - isExtraDep = stackPackage ^. spExtraDep pconf = PackageConfig - { enableCheck = if isExtraDep then conf ^. spcDoCheckStackage else conf ^. spcDoCheckPackages - , enableHaddock = if isExtraDep then conf ^. spcDoHaddockStackage else conf ^. spcDoHaddockPackages } + { enableCheck = conf ^. spcDoCheckPackages + , enableHaddock = conf ^. spcDoHaddockPackages } return $ finalizePackage pkg pconf drv genericPackageDerivation diff --git a/src/Distribution/Nixpkgs/Haskell/Stack/PrettyPrinting.hs b/src/Distribution/Nixpkgs/Haskell/Stack/PrettyPrinting.hs --- a/src/Distribution/Nixpkgs/Haskell/Stack/PrettyPrinting.hs +++ b/src/Distribution/Nixpkgs/Haskell/Stack/PrettyPrinting.hs @@ -15,6 +15,7 @@ import Distribution.Version (Version) import qualified Language.Nix.FilePath as Nix import Language.Nix.PrettyPrinting as PP +import Stack.Config (StackResolver, unStackResolver) data OverrideConfig = OverrideConfig @@ -29,6 +30,11 @@ systemNixpkgs :: Doc systemNixpkgs = "" +renderNixpkgs :: FilePath -> Doc +renderNixpkgs nixpkgsPath + | fromString nixpkgsPath == systemNixpkgs = systemNixpkgs + | otherwise = disp $ (fromString nixpkgsPath :: Nix.FilePath) + hasField :: Lens' a (Maybe b) -> a -> Bool hasField p = views p isJust @@ -49,14 +55,9 @@ ["import ", disp (fromString path :: Nix.FilePath), "{ inherit pkgs haskellLib; }"] overrideHaskellPackages :: OverrideConfig -> NonEmpty Derivation -> Doc -overrideHaskellPackages oc packages = - let - nixpkgs = if oc ^. ocNixpkgs . to fromString == systemNixpkgs - then systemNixpkgs - else (disp . (fromString :: FilePath -> Nix.FilePath)) (oc ^. ocNixpkgs) - in vcat +overrideHaskellPackages oc packages = vcat [ funargs - [ "nixpkgs ? import " <> nixpkgs <> " {}" + [ "nixpkgs ? import " <> renderNixpkgs (oc ^. ocNixpkgs) <> " {}" ] , "" , "with nixpkgs;" @@ -87,32 +88,52 @@ , "}" ] +overrideStackage :: StackResolver -> FilePath -> NonEmpty Derivation -> Doc +overrideStackage stackResolver nixpkgsPath packages = vcat + [ funargs + [ "nixpkgs ? import " <> renderNixpkgs nixpkgsPath <> " {}" + ] + , "" + , "let" + , nest 2 $ vcat + [ "stackPackages =" + , nest 2 $ overridePackages packages <> semi + ] + , "in nixpkgs.haskell.packages.stackage." <> toNixStackResolver stackResolver <> ".override {" + , nest 2 + $ attr "packageSetConfig" "self: super: stackPackages { inherit (nixpkgs) pkgs stdenv; inherit (self) callPackage; } super" + , "}" + ] + pPrintHaskellPackages :: OverrideConfig -> Doc -pPrintHaskellPackages oc = - let - nixpkgs = if oc ^. ocNixpkgs . to fromString == systemNixpkgs - then systemNixpkgs - else (disp . (fromString :: FilePath -> Nix.FilePath)) (oc ^. ocNixpkgs) - in vcat +pPrintHaskellPackages oc = vcat [ funargs - [ "nixpkgs ? import " <> nixpkgs <> " {}" + [ "callPackage" + , "pkgs" + , "overrides ? (self: super: {})" + , "packageSetConfig ? (sefl: super: {})" ] , "" - , "with nixpkgs; let" + , "let" , nest 2 $ vcat - [ attr "haskellLib" "callPackage (nixpkgs.path + /pkgs/development/haskell-modules/lib.nix) {}" + [ attr "haskellLib" "pkgs.haskell.lib" ] - , "in callPackage (nixpkgs.path + /pkgs/development/haskell-modules) {" + , "in callPackage () {" , nest 2 $ vcat [ attr "ghc" ("pkgs.haskell.compiler." <> toNixGhcVersion (oc ^. ocGhc)) , attr "compilerConfig" . importStackageConfig $ oc ^. ocStackageConfig , attr "initialPackages" . importStackagePackages $ oc ^. ocStackagePackages , attr "configurationCommon" "if builtins.pathExists ./configuration-common.nix then import ./configuration-common.nix else args: self: super: {}" - , "inherit haskellLib;" + , "inherit haskellLib overrides packageSetConfig;" ] , "}" ] toNixGhcVersion :: Version -> Doc -toNixGhcVersion = - (<>) "ghc" . text . L.filter (/= '.') . show . disp +toNixGhcVersion = (<>) "ghc" . toNixVersion . show . disp + +toNixStackResolver :: StackResolver -> Doc +toNixStackResolver = toNixVersion . unStackResolver + +toNixVersion :: String -> Doc +toNixVersion = text . L.filter (/= '.') diff --git a/src/LtsHaskell.hs b/src/LtsHaskell.hs --- a/src/LtsHaskell.hs +++ b/src/LtsHaskell.hs @@ -1,4 +1,3 @@ -{-# LANGUAGE CPP #-} module LtsHaskell where import AllCabalHashes @@ -24,6 +23,7 @@ import Stackage.Types (SystemInfo(..)) import System.FilePath as Path +import qualified Distribution.Nixpkgs.Haskell.Hackage as DB import qualified Data.Yaml as Yaml import qualified Data.Map as Map import qualified Data.Set as Set @@ -47,21 +47,17 @@ (error (display pkgId ++ ": meta data has no SHA256 hash for the tarball")) (view (mHashes . at "SHA256") meta) source = DerivationSource "url" ("mirror://hackage/" ++ display pkgId ++ ".tar.gz") "" tarballSHA256 -#if MIN_VERSION_cabal2nix(2,6,0) return $ Package source False pkgDesc -#else - return $ Package source pkgDesc -#endif -getPackageFromDb :: Maybe HackageDb -> PackageIdentifier -> IO Package -getPackageFromDb mHackageDb pkgId = - getStackPackageFromDb mHackageDb +getPackageFromDb :: DB.HackageDB -> PackageIdentifier -> IO Package +getPackageFromDb hackageDb pkgId = + getStackPackageFromDb hackageDb $ StackPackage (HackagePackage (T.pack $ Text.display pkgId)) True Nothing -loadPackage :: Maybe HackageDb -> FilePath -> Maybe SHA1Hash -> PackageIdentifier -> IO Package -loadPackage mHackageDb allCabalHashesPath mSha1Hash pkgId = +loadPackage :: DB.HackageDB -> FilePath -> Maybe SHA1Hash -> PackageIdentifier -> IO Package +loadPackage hackageDb allCabalHashesPath mSha1Hash pkgId = getPackageFromRepo allCabalHashesPath mSha1Hash pkgId - `catchIOError` const (getPackageFromDb mHackageDb pkgId) + `catchIOError` const (getPackageFromDb hackageDb pkgId) ghcCompilerInfo :: Version -> CompilerInfo ghcCompilerInfo v = CompilerInfo @@ -77,18 +73,18 @@ maybe False (`withinRange` versionRange) $ Map.lookup depName packageVersions buildPackageSetConfig - :: Maybe HackageDb + :: DB.HackageDB -> FilePath -> FilePath -> BuildPlan -> IO PackageSetConfig -buildPackageSetConfig mHackageDb optAllCabalHashes optNixpkgsRepository buildPlan = do +buildPackageSetConfig hackageDb optAllCabalHashes optNixpkgsRepository buildPlan = do nixpkgs <- readNixpkgPackageMap optNixpkgsRepository Nothing let systemInfo = bpSystemInfo buildPlan packageVersions = fmap ppVersion (bpPackages buildPlan) `Map.union` siCorePackages systemInfo return PackageSetConfig - { packageLoader = loadPackage mHackageDb optAllCabalHashes + { packageLoader = loadPackage hackageDb optAllCabalHashes , targetPlatform = Platform (siArch systemInfo) (siOS systemInfo) , targetCompiler = ghcCompilerInfo (siGhcVersion systemInfo) , nixpkgsResolver = resolve (Map.map (Set.map (over path ("pkgs":))) nixpkgs) diff --git a/src/Runner.hs b/src/Runner.hs --- a/src/Runner.hs +++ b/src/Runner.hs @@ -3,13 +3,17 @@ import Control.Lens import Data.Foldable as F import Data.List as L +import Data.Maybe (fromMaybe) +import Data.Semigroup ((<>)) import Distribution.Nixpkgs.Haskell.FromStack import Distribution.Nixpkgs.Haskell.FromStack.Package +import Distribution.Nixpkgs.Haskell.Hackage as DB +import Distribution.Nixpkgs.Haskell.PackageSourceSpec (loadHackageDB) import Distribution.Nixpkgs.Haskell.Stack import Distribution.Nixpkgs.Haskell.Stack.PrettyPrinting as PP import Distribution.Version (Version) import Distribution.Compiler (AbiTag(..), unknownCompilerInfo) -import Distribution.Package (mkPackageName, pkgName) +import Distribution.Package (PackageName, mkPackageName, pkgName) import Distribution.Text as Text (display) import Language.Nix as Nix import Options.Applicative @@ -25,84 +29,95 @@ import qualified Data.Map as Map import qualified LtsHaskell as LH - run :: IO () run = do opts <- execParser pinfo + hackageDb <- loadHackageDB (opts ^. optHackageDb) Nothing + let + -- With optparse-applicative it seems there's no way to make + -- 'StackageOptions' optional for Stackage override and required in + -- other cases + missingErr = error . ("Missing required parameter: " <>) + sopts = StackageOptions + (fromMaybe (missingErr "--all-cabal-hashes") (opts ^. optAllCabalHashesRepo) ) + (fromMaybe (missingErr "--lts-haskell") (opts ^. optLtsHaskellRepo)) + case opts ^. optConfigOrigin of - -- Generate build derivation from stack.yaml file - OriginStackYaml stackYaml -> do + -- Generate Stackage packages from stack.yaml resolver + OriginStackYaml stackYaml buildTarget -> do stackConf <- either fail pure =<< readStackConfig stackYaml - let buildPlanFile = LH.buildPlanFilePath (opts ^. optLtsHaskellRepo) (stackConf ^. scResolver) - buildPlan <- LH.loadBuildPlan buildPlanFile - packageSetConfig <- LH.buildPackageSetConfig - (opts ^. optHackageDb) - (opts ^. optAllCabalHashesRepo) - (opts ^. optNixpkgsRepository) - buildPlan let - overrideConfig = mkOverrideConfig opts (siGhcVersion $ bpSystemInfo buildPlan) + stackResolver = stackConf ^. scResolver stackPackagesConfig = mkStackPackagesConfig opts - stackConfPackages <- traverse (packageDerivation stackPackagesConfig (opts ^. optHackageDb)) + stackConfPackages <- traverse (fmap mkNode . packageDerivation stackPackagesConfig hackageDb) $ stackConf ^. scPackages - let - reachable = Set.map mkPackageName - $ F.foldr1 Set.union - $ nodeDepends . mkNode <$> stackConfPackages - s2nLoader mHash pkgId = - if pkgName pkgId `Set.member` reachable - then packageLoader packageSetConfig Nothing pkgId - else packageLoader packageSetConfig mHash pkgId - s2nPackageSetConfig = packageSetConfig { packageLoader = s2nLoader } - s2nPackageConfig = PackageConfig - { enableCheck = opts ^. optDoCheckStackage - , enableHaddock = opts ^. optDoHaddockStackage } - stackagePackages <- traverse (uncurry (buildNode s2nPackageSetConfig s2nPackageConfig)) - $ Map.toAscList (bpPackages buildPlan) - let - -- Nixpkgs generic-builder puts hscolour on path for all libraries - withHscolour pkgs = - let hscolour = F.find ((== "hscolour") . nodeName) stackagePackages - in maybe pkgs (`Set.insert` pkgs) hscolour - -- Find all reachable dependencies in stackage set to stick into - -- stackage packages file. This is performed on the full stackage - -- set rather than pruning stackage packages beforehand because - -- stackage does not concern itself with build tools while cabal2nix - -- does: pruning only after generating full set of packages allows - -- us to make sure all those extra dependencies are explicitly - -- listed as well. - nodes = case opts ^. optOutPackagesClosure of - True -> Set.toAscList - $ withHscolour - $ flip reachableDependencies stackagePackages - -- Originally reachable nodes are root nodes - $ L.filter (\n -> mkPackageName (nodeName n) `Set.member` reachable) stackagePackages - False -> stackagePackages - writeOutFile buildPlanFile (opts ^. optOutStackagePackages) - $ pPrintOutPackages (view nodeDerivation <$> nodes) - writeOutFile buildPlanFile (opts ^. optOutStackageConfig) - $ pPrintOutConfig (bpSystemInfo buildPlan) nodes - writeOutFile (stackYaml ^. syFilePath) (opts ^. optOutDerivation) - $ PP.overrideHaskellPackages overrideConfig stackConfPackages + case buildTarget of + TargetStackageClosure -> do + (buildPlanFile, buildPlan, packageSetConfig) <- loadStackageBuildPlan hackageDb stackResolver opts sopts + let + overrideConfig = mkOverrideConfig opts (siGhcVersion $ bpSystemInfo buildPlan) + stackageSetConfig = if opts ^. optTweaks . tExtraDepsRevisionLatest + then extraDepsLatestConfig stackConfPackages packageSetConfig + else packageSetConfig + stackagePackages <- buildStackagePackages stackageSetConfig buildPlan + let + reachable = reachableDeps stackConfPackages + -- Nixpkgs generic-builder puts hscolour on path for all libraries + withHscolour pkgs = + let hscolour = F.find ((== "hscolour") . nodeName) stackagePackages + in maybe pkgs (`Set.insert` pkgs) hscolour + -- Find all reachable dependencies in stackage set to stick into + -- stackage packages file. This is performed on the full stackage + -- set rather than pruning stackage packages beforehand because + -- stackage does not concern itself with build tools while cabal2nix + -- does: pruning only after generating full set of packages allows + -- us to make sure all those extra dependencies are explicitly + -- listed as well. + nodes = Set.toAscList + $ withHscolour + $ flip reachableDependencies stackagePackages + -- Originally reachable nodes are root nodes + $ L.filter (\n -> mkPackageName (nodeName n) `Set.member` reachable) stackagePackages + writeOutFile buildPlanFile (opts ^. optOutStackagePackages) + $ pPrintOutPackages (view nodeDerivation <$> nodes) + writeOutFile buildPlanFile (opts ^. optOutStackageConfig) + $ pPrintOutConfig (bpSystemInfo buildPlan) nodes + writeOutFile (stackYaml ^. syFilePath) (opts ^. optOutDerivation) + $ PP.overrideHaskellPackages overrideConfig (view nodeDerivation <$> stackConfPackages) + TargetStackagePackages -> do + (buildPlanFile, buildPlan, packageSetConfig) <- loadStackageBuildPlan hackageDb stackResolver opts sopts + let + overrideConfig = mkOverrideConfig opts (siGhcVersion $ bpSystemInfo buildPlan) + stackageSetConfig = if opts ^. optTweaks . tExtraDepsRevisionLatest + then extraDepsLatestConfig stackConfPackages packageSetConfig + else packageSetConfig + -- stackageLoader mHash pkgId = + -- if pkgName pkgId `Set.member` reachable + -- then packageLoader packageSetConfig Nothing pkgId + -- else packageLoader packageSetConfig mHash pkgId + -- stackageSetConfig = packageSetConfig { packageLoader = stackageLoader } + nodes <- buildStackagePackages stackageSetConfig buildPlan + writeOutFile buildPlanFile (opts ^. optOutStackagePackages) + $ pPrintOutPackages (view nodeDerivation <$> nodes) + writeOutFile buildPlanFile (opts ^. optOutStackageConfig) + $ pPrintOutConfig (bpSystemInfo buildPlan) nodes + writeOutFile (stackYaml ^. syFilePath) (opts ^. optOutDerivation) + $ PP.overrideHaskellPackages overrideConfig (view nodeDerivation <$> stackConfPackages) + + TargetStackageOverride -> + writeOutFile (stackYaml ^. syFilePath) (opts ^. optOutDerivation) $ + PP.overrideStackage + (stackConf ^. scResolver) + (opts ^. optNixpkgsRepository) + (view nodeDerivation <$> stackConfPackages) + -- Generate Stackage packages from resolver OriginResolver stackResolver -> do - let - buildPlanFile = LH.buildPlanFilePath (opts ^. optLtsHaskellRepo) stackResolver - packageConfig = PackageConfig - { enableCheck = True - , enableHaddock = True } - buildPlan <- LH.loadBuildPlan buildPlanFile - packageSetConfig <- LH.buildPackageSetConfig - (opts ^. optHackageDb) - (opts ^. optAllCabalHashesRepo) - (opts ^. optNixpkgsRepository) - buildPlan - nodes <- traverse (uncurry (buildNode packageSetConfig packageConfig)) - $ Map.toAscList (bpPackages buildPlan) + (buildPlanFile, buildPlan, packageSetConfig) <- loadStackageBuildPlan hackageDb stackResolver opts sopts + nodes <- buildStackagePackages packageSetConfig buildPlan let overrideConfig = mkOverrideConfig opts (siGhcVersion $ bpSystemInfo buildPlan) - writeOutFile buildPlanFile (opts ^. optOutStackagePackages) $ pPrintOutPackages (view nodeDerivation <$> nodes) writeOutFile buildPlanFile (opts ^. optOutStackageConfig) @@ -110,6 +125,43 @@ writeOutFile buildPlanFile (opts ^. optOutDerivation) $ PP.pPrintHaskellPackages overrideConfig +loadStackageBuildPlan :: DB.HackageDB -> StackResolver -> Options -> StackageOptions -> IO (FilePath, BuildPlan, PackageSetConfig) +loadStackageBuildPlan hackageDb stackResolver opts sopts = do + let buildPlanFile = LH.buildPlanFilePath (sopts ^. soptLtsHaskellRepo) stackResolver + buildPlan <- LH.loadBuildPlan buildPlanFile + packageSetConfig <- LH.buildPackageSetConfig + hackageDb + (sopts ^. soptAllCabalHashesRepo) + (opts ^. optNixpkgsRepository) + buildPlan + return (buildPlanFile, buildPlan, packageSetConfig) + +buildStackagePackages :: PackageSetConfig -> BuildPlan -> IO [Node] +buildStackagePackages packageSetConfig buildPlan = do + let + stackagePackageConfig = PackageConfig + { enableCheck = True + , enableHaddock = True } + traverse (uncurry (buildNode packageSetConfig stackagePackageConfig)) + $ Map.toAscList (bpPackages buildPlan) + +reachableDeps :: Traversable t => t Node -> Set.Set PackageName +reachableDeps = Set.map mkPackageName + . F.foldr1 Set.union + . fmap nodeDepends + +-- Return 'PackageSetConfig' with package loader that loads latest Hackage +-- revison for direct (reachable) dependencies of input packages +extraDepsLatestConfig :: Traversable t => t Node -> PackageSetConfig -> PackageSetConfig +extraDepsLatestConfig packages packageSetConfig = + let + reachable = reachableDeps packages + stackageLoader mHash pkgId = + if pkgName pkgId `Set.member` reachable + then packageLoader packageSetConfig Nothing pkgId + else packageLoader packageSetConfig mHash pkgId + in packageSetConfig { packageLoader = stackageLoader } + writeOutFile :: Show source => source -> FilePath -> Doc -> IO () writeOutFile source filePath contents = withFile filePath WriteMode $ \h -> do @@ -131,6 +183,4 @@ , _spcTargetCompiler = unknownCompilerInfo (opts ^. optCompilerId) NoAbiTag , _spcFlagAssignment = [] , _spcDoCheckPackages = opts ^. optDoCheckPackages - , _spcDoHaddockPackages = opts ^. optDoHaddockPackages - , _spcDoCheckStackage = opts ^. optDoCheckStackage - , _spcDoHaddockStackage = opts ^. optDoHaddockStackage } + , _spcDoHaddockPackages = opts ^. optDoHaddockPackages } diff --git a/src/Runner/Cli.hs b/src/Runner/Cli.hs --- a/src/Runner/Cli.hs +++ b/src/Runner/Cli.hs @@ -5,38 +5,56 @@ import Data.Text as T import qualified Distribution.Compat.ReadP as P import Distribution.Compiler as Compiler -import Distribution.Nixpkgs.Haskell.Stack import Distribution.System as System import qualified Distribution.Text as Text import Options.Applicative as Opts import Paths_stackage2nix ( version ) import Stack.Config import Stack.Types -import System.Environment import System.FilePath +data Tweaks = Tweaks + { _tExtraDepsRevisionLatest :: Bool + } deriving (Show) + +makeLenses ''Tweaks + +-- | What will be generated +data Target + = TargetStackagePackages + | TargetStackageClosure + | TargetStackageOverride + deriving (Show) + +makePrisms ''Target + -- | Source that would be used to produce target derivation data ConfigOrigin = OriginResolver StackResolver -- ^ Resolver to generate Stackage LTS - | OriginStackYaml StackYaml + | OriginStackYaml StackYaml Target -- ^ Stack config to generate build derivation deriving (Show) makePrisms ''ConfigOrigin +data StackageOptions = StackageOptions + { _soptAllCabalHashesRepo :: FilePath + , _soptLtsHaskellRepo :: FilePath + } deriving (Show) + +makeLenses ''StackageOptions + data Options = Options - { _optAllCabalHashesRepo :: FilePath - , _optLtsHaskellRepo :: FilePath + { _optAllCabalHashesRepo :: Maybe FilePath + , _optLtsHaskellRepo :: Maybe FilePath , _optOutStackagePackages :: FilePath , _optOutStackageConfig :: FilePath - , _optOutPackagesClosure :: Bool , _optOutDerivation :: FilePath , _optDoCheckPackages :: Bool , _optDoHaddockPackages :: Bool - , _optDoCheckStackage :: Bool - , _optDoHaddockStackage :: Bool - , _optHackageDb :: (Maybe HackageDb) + , _optTweaks :: Tweaks + , _optHackageDb :: Maybe FilePath , _optNixpkgsRepository :: FilePath , _optCompilerId :: CompilerId , _optPlatform :: Platform @@ -45,29 +63,26 @@ makeLenses ''Options -envStackYaml :: IO (Maybe StackYaml) -envStackYaml = fmap mkStackYaml <$> lookupEnv "STACK_YAML" - mkStackYaml :: FilePath -> StackYaml mkStackYaml p = case splitFileName p of (dir, "") -> StackYaml dir "stack.yaml" (dir, ".") -> StackYaml dir "stack.yaml" (_ , "..") -> StackYaml p "stack.yaml" - (dir, file) -> StackYaml dir file + (dir, file) -> case splitExtension file of + (_, "") -> StackYaml p "stack.yaml" + _ -> StackYaml dir file options :: Parser Options options = Options - <$> allCabalHashesRepo - <*> ltsHaskellRepo + <$> optional allCabalHashesRepo + <*> optional ltsHaskellRepo <*> outStackagePackages <*> outStackageConfig - <*> outPackagesClosure <*> outDerivation <*> doCheckPackages <*> doHaddockPackages - <*> doCheckStackage - <*> doHaddockStackage - <*> optional hackageDb + <*> tweaks + <*> optional optionHackageDb <*> nixpkgsRepository <*> compilerId <*> platform @@ -117,11 +132,6 @@ <> value "configuration-packages.nix" <> showDefaultWith id) -outPackagesClosure :: Parser Bool -outPackagesClosure = flag True False - ( long "no-packages-closure" - <> help "produce full set of stackage packages, not just dependencies" ) - outDerivation :: Parser FilePath outDerivation = option str ( long "out-derivation" @@ -140,16 +150,24 @@ ( long "no-haddock" <> help "disable haddock for project packages") -doCheckStackage :: Parser Bool -doCheckStackage = switch - ( long "do-check-stackage" - <> help "enable tests for Stackage packages" ) +withStackage :: Parser () +withStackage = flag' () + ( long "with-stackage" + <> help "generate full Stackage" ) -doHaddockStackage :: Parser Bool -doHaddockStackage = switch - ( long "do-haddock-stackage" - <> help "enable haddock for Stackage packages" ) +withStackageClosure :: Parser () +withStackageClosure = flag' () + ( long "with-stackage-closure" + <> help "generate Stackage subset containing only build dependencies" ) +extraDepsRevisionLatest :: Parser Bool +extraDepsRevisionLatest = switch + ( long "extra-deps-revision-latest" + <> help "Changes algorithm for the dependencies generation of the stack.yaml extra-deps. Default one is to generate exact revision defined in the Stackage config. With this flag enabled, direct dependencies of the stack.yaml extra-deps will be generated with the latest revision available on Hackage. This may help when Cabal fails to resolve dependencies of extra-deps, and dependences are fixed in the latest revision" ) + +tweaks :: Parser Tweaks +tweaks = Tweaks <$> extraDepsRevisionLatest + resolver :: Parser StackResolver resolver = StackResolver <$> option text ( long "resolver" @@ -160,18 +178,24 @@ ( metavar "STACK_YAML" <> help "path to stack.yaml file or directory" ) +target :: Parser Target +target = + withStackage *> pure TargetStackagePackages <|> + withStackageClosure *> pure TargetStackageClosure <|> + pure TargetStackageOverride + configOrigin :: Parser ConfigOrigin -configOrigin = OriginResolver <$> resolver - <|> OriginStackYaml <$> stackYamlArg +configOrigin = + OriginStackYaml <$> stackYamlArg <*> target <|> + OriginResolver <$> resolver --- inherited from cabal2nix +-- required for cabal2nix -hackageDb :: Parser HackageDb -hackageDb = HackageDb <$> - option text - ( long "hackage-db" - <> metavar "HACKAGE_DB" - <> help "path to the local hackage db in tar format" ) +optionHackageDb :: Parser FilePath +optionHackageDb = option str + ( long "hackage-db" + <> metavar "HACKAGE_DB" + <> help "path to the local hackage db in tar format" ) cabalFlag :: Parser CabalFlag cabalFlag = CabalFlag <$> diff --git a/stackage2nix.cabal b/stackage2nix.cabal --- a/stackage2nix.cabal +++ b/stackage2nix.cabal @@ -1,5 +1,5 @@ name: stackage2nix -version: 0.4.0 +version: 0.5.0 synopsis: Convert Stack files into Nix build instructions. homepage: https://github.com/typeable/stackage2nix#readme license: BSD3 @@ -31,11 +31,11 @@ , LtsHaskell , Paths_stackage2nix build-depends: base > 4.7 && < 5 - , Cabal > 1.24 + , Cabal > 2 , QuickCheck , aeson , bytestring - , cabal2nix >= 2.5 + , cabal2nix >= 2.7.2 , containers , deepseq , distribution-nixpkgs >= 1.1 @@ -59,7 +59,7 @@ , RecordWildCards , TemplateHaskell default-language: Haskell2010 - ghc-options: -Wall -funbox-strict-fields + ghc-options: -Wall executable stackage2nix hs-source-dirs: stackage2nix