diff --git a/README.md b/README.md --- a/README.md +++ b/README.md @@ -5,6 +5,43 @@ This is a build system for ATS written in Haskell and configured with Dhall. It is not yet stable. +## Features & Non-Features + +Things that `atspkg` will do for you: + + * Dramatically simplify CI for ATS projects + * Simplify distribution of your project + * Enable Haskell builds that depend on ATS code + * Enable ATS builds that depend on Haskell code + * Ensure reproducible builds via pinned compiler versions + * Track all file dependencies + * Make contributing to your projects easier + * Run builds in parallel (like `make`) + * Handle flags and libraries for garbage collection when specified + * Install `patscc` and other ATS tooling + +Things that `atspkg` will not do for you: + + * Dependency resolution (this is planned) + * Give you the full flexibility of the C/ATS ecosystem + * Integrate with other ecosystems + * Provide a centralized package repository + * Library builds (this is planned) + * Offer a common architecture for package builds + * Cache builds locally (like `nix` or `cabal`) + +### Example + +As an example, the following two lines will install `getkb`: + +```bash +curl -sSl https://raw.githubusercontent.com/vmchale/atspkg/master/bash/install.sh | bash -s +atspkg remote https://github.com/vmchale/getkb/archive/master.zip +``` + +As you can see, this greatly simplifies distribution and testing of programs +written in ATS. + ## Installation ### Script @@ -35,101 +72,20 @@ Note that `$HOME/.local/bin` will need to be on your `PATH`. -## Examples - -`atspkg` is configured with -[Dhall](https://hackage.haskell.org/package/dhall/docs/Dhall-Tutorial.html). You -may wish to read the Dhall tutorial first, but you do not need to fully -understand everything to get started. - -### Project Templates - -You can use [pi](https://github.com/vmchale/project-init) with the builtin `ats` -template as follows: - -``` -pi new ats cool-project -``` - -You can then build with `atspkg build` or install with `atspkg install`. +## Global Configuration -Alternately, you can start with a templated Haskell library calling ATS code: +`atspkg` is configured via a file in `~/.config/atspkg/config.dhall`. You can +set custom package set as follows: ``` -pi git vmchale/haskell-ats ambitious-project -``` - -which can be built with `atspkg build` followed by `cabal new-build`. - -### Building a Binary Package - -The minimal configuration for a package with a binary target is as follows: - -```dhall -let pkg = https://raw.githubusercontent.com/vmchale/atspkg/master/pkgs/default.dhall -in -let dbin = https://raw.githubusercontent.com/vmchale/atspkg/master/pkgs/default-bin.dhall - -in pkg // - { bin = - [ dbin // - { src = "src/program.dats" - , target = "target/program" - } - ] +let cfg = + { defaultPkgs = "https://raw.githubusercontent.com/vmchale/atspkg/master/pkgs/pkg-set.dhall" } -``` -You need only specify the source file and the target; `atspkg` will parse your -ATS source files and track them (it will not track included C). - -### Building a Haskell Library - -You can see an example [here](https://github.com/vmchale/fast-arithmetic). You -can configure the ATS side of things as follows: - -``` -let pkg = https://raw.githubusercontent.com/vmchale/atspkg/master/pkgs/default.dhall - -in pkg // - { atsSource = [ "ats-src/ambitious-project.dats" ] } +in cfg ``` -This just tells `atspkg` to look for a source file called -`ats-src/ambitious-project.dats`, which will be compiled to -`ambitious-project.c` in the default directory (i.e. `cbits`). You can then -call the generated code just as you would call C. - -You may want to consider -[ats-setup](http://hackage.haskell.org/package/ats-setup) as well if you are -packaging the Haskell for distribution. - -### Calling Haskell from ATS - -You can see a demo [here](https://github.com/vmchale/fast-arithmetic). -Currently, there is not generic `Storable` instance that works with ATS, so the -process is a bit more involved than is ideal. `atspkg` has abilities similar to -[hs2ats](http://hackage.haskell.org/package/hs2ats), which means that you can -usually generate ATS types based on the Haskell types. - -The following is a minimal example of a configuration file: - -```dhall -let pkg = https://raw.githubusercontent.com/vmchale/atspkg/master/pkgs/default.dhall -in -let dbin = https://raw.githubusercontent.com/vmchale/atspkg/master/pkgs/default-bin.dhall +## Examples -in pkg // - { bin = - [ - dbin // - { src = "src/project.dats" - , target = "target/project" - , hsDeps = [ { cabalFile = "hs/foreign.cabal", objectFile = "hs/Foreign.o" } ] - , hs2ats = [ { hs = "hs/Foreign.hs", ats = ".atspkg/hs2ats/gen.sats" } ] - } - ] - , ccompiler = "ghc-8.2.2" - , cflags = ["-package-db", "hs/dist-newstyle/packagedb/ghc-8.2.2/", "-optc-O2", "-optc-flto", "-optc-mtune=native", "hs/Foreign"] - } -``` +You can find several examples with explanation +[here](https://github.com/vmchale/atspkg/blob/master/EXAMPLES.md) diff --git a/ats-pkg.cabal b/ats-pkg.cabal --- a/ats-pkg.cabal +++ b/ats-pkg.cabal @@ -1,5 +1,5 @@ name: ats-pkg -version: 2.3.0.6 +version: 2.4.0.0 synopsis: A build tool for ATS description: A collection of scripts to simplify building ATS projects. homepage: https://github.com/vmchale/atspkg#readme @@ -40,10 +40,12 @@ , Language.ATS.Package.Build , Language.ATS.Package.Upgrade , Language.ATS.Package.Tools + , Language.ATS.Package.Config , Language.ATS.Package.PackageSet build-depends: base >= 4.7 && < 5 , http-client , bytestring + , file-embed , shake , bzlib , lzma diff --git a/config.dhall b/config.dhall --- a/config.dhall +++ b/config.dhall @@ -1,6 +1,5 @@ let cfg = { defaultPkgs = "https://raw.githubusercontent.com/vmchale/atspkg/master/pkgs/pkg-set.dhall" - , solverIters = 15 } in cfg diff --git a/src/Language/ATS/Package/Build.hs b/src/Language/ATS/Package/Build.hs --- a/src/Language/ATS/Package/Build.hs +++ b/src/Language/ATS/Package/Build.hs @@ -31,6 +31,7 @@ import Development.Shake.Man import Dhall hiding (bool, maybe) import Language.ATS.Package.Compiler +import Language.ATS.Package.Config import Language.ATS.Package.Dependency import Language.ATS.Package.Type hiding (version) import Paths_ats_pkg @@ -135,6 +136,7 @@ , shakeRebuild = foldMap g [ (rb, [(RebuildNow, ".atspkg/config")]) , (rba, (RebuildNow ,) <$> rs) ] + , shakeChange = ChangeModtimeAndDigestInput } where g (b, ts) = bool mempty ts b @@ -200,13 +202,15 @@ unless (rs == ["clean"]) $ do - checkDeps <- liftIO $ doesFileExist ".atspkg/config" - - let cdps = if any gcBin bs then "gc" : cds else cds + let cdps = if f bs || f ts then "gc" : cds else cds where f = any gcBin - liftIO $ fetchDeps (ccFromString cc') setup (TL.unpack <$> ds) (TL.unpack <$> cdps) checkDeps >> stopGlobalPool + mkUserConfig - want [".atspkg/config"] + ".atspkg/deps" %> \out -> do + (_, cfgBin') <- cfgBin + need [ cfgBin' ] + liftIO $ fetchDeps (ccFromString cc') setup (TL.unpack <$> ds) (TL.unpack <$> cdps) cfgBin' False >> stopGlobalPool + liftIO $ writeFile out "" let bins = TL.unpack . target <$> bs setTargets rs bins mt @@ -217,7 +221,7 @@ where g (Bin s t ls hs' atg gc' cSrc) = atsBin - (BinaryTarget cc' (TL.unpack <$> cf) (ATSToolConfig v v' False) gc' (TL.unpack <$> ls) (TL.unpack s) hs' (unpackBoth . asTuple <$> atg) (TL.unpack t) (TL.unpack <$> cSrc)) + (BinaryTarget (TL.unpack <$> cf) (ATSToolConfig v v' False (ccFromString cc')) gc' (TL.unpack <$> ls) (TL.unpack s) hs' (unpackBoth . asTuple <$> atg) (TL.unpack t) (TL.unpack <$> cSrc) [".atspkg/deps", ".atspkg/config"] (Binary False)) cDepsRules = unless (null as) $ do let cedar = TL.unpack cdir @@ -225,7 +229,7 @@ targets = fmap (((cedar <> "/") <>) . (-<.> "c") . takeBaseName . TL.unpack) as want targets hasPF <- patsFilter - mapM_ (cgen $ ATSToolConfig v v' hasPF) atsSourceDirs + mapM_ (cgen $ ATSToolConfig v v' hasPF (ccFromString cc')) atsSourceDirs cc' = maybe (TL.unpack ccLocal) (<> "-gcc") tgt diff --git a/src/Language/ATS/Package/Config.hs b/src/Language/ATS/Package/Config.hs new file mode 100644 --- /dev/null +++ b/src/Language/ATS/Package/Config.hs @@ -0,0 +1,56 @@ +{-# LANGUAGE DeriveAnyClass #-} +{-# LANGUAGE DeriveGeneric #-} +{-# LANGUAGE TemplateHaskell #-} + +module Language.ATS.Package.Config ( UserConfig (..) + , mkUserConfig + , cfgBin + ) where + +import Control.Arrow +import Control.Monad.IO.Class +import Data.Binary +import qualified Data.ByteString.Lazy as BSL +import Data.FileEmbed +import qualified Data.Text.Lazy as TL +import Development.Shake hiding (getEnv) +import Dhall +import System.Directory (createDirectoryIfMissing) +import System.Environment (getEnv) + +newtype UserConfig = UserConfig { defaultPkgs :: Text + } deriving (Generic, Interpret, Binary) + +cfgFile :: String +cfgFile = $(embedStringFile "config.dhall") + +defaultFileConfig :: FilePath -> IO () +defaultFileConfig p = do + let dir = p ++ "/.config/atspkg" + createDirectoryIfMissing True dir + writeFile (dir ++ "/config.dhall") cfgFile + +cfgBin :: (MonadIO m) => m (FilePath, FilePath) +cfgBin = liftIO io + where io = (id &&& (++ "/.atspkg/config")) <$> getEnv "HOME" + +mkUserConfig :: Rules () +mkUserConfig = do + + (h, cfgBin') <- cfgBin + let cfg = h ++ "/.config/atspkg/config.dhall" + + want [cfgBin'] + + readUserConfig h cfg + + cfgBin' %> \_ -> do + need [cfg] + cfgContents <- liftIO $ input auto (TL.pack cfg) + liftIO $ BSL.writeFile cfgBin' (encode (cfgContents :: UserConfig)) + +readUserConfig :: FilePath -> FilePath -> Rules () +readUserConfig h cfg = do + want [cfg] + + cfg %> \_ -> liftIO (defaultFileConfig h) diff --git a/src/Language/ATS/Package/Dependency.hs b/src/Language/ATS/Package/Dependency.hs --- a/src/Language/ATS/Package/Dependency.hs +++ b/src/Language/ATS/Package/Dependency.hs @@ -13,12 +13,15 @@ import Control.Concurrent.ParallelIO.Global import Control.Lens import Control.Monad +import Data.Binary (decode) import Data.ByteString.Lazy (ByteString) +import qualified Data.ByteString.Lazy as BSL import Data.Maybe (fromMaybe) import Data.Semigroup (Semigroup (..)) import qualified Data.Text.Lazy as TL import Development.Shake.ATS import Dhall +import Language.ATS.Package.Config import Language.ATS.Package.Error import Language.ATS.Package.PackageSet import Language.ATS.Package.Type @@ -33,15 +36,17 @@ -> [IO ()] -- ^ Setup steps that can be performed concurrently -> [String] -- ^ ATS dependencies -> [String] -- ^ C Dependencies + -> FilePath -- ^ Path to configuration file -> Bool -- ^ Whether to perform setup anyhow. -> IO () -fetchDeps cc' setup' deps cdeps b' = +fetchDeps cc' setup' deps cdeps cfgPath b' = unless (null deps && null cdeps && b') $ do - deps' <- join <$> setBuildPlan "ats" deps + pkgSet <- TL.unpack . defaultPkgs . decode <$> BSL.readFile cfgPath + deps' <- join <$> setBuildPlan "ats" pkgSet deps putStrLn "Checking ATS dependencies..." d <- (<> "lib/") <$> pkgHome cc' let libs' = fmap (buildHelper False) deps' - cdeps' <- join <$> setBuildPlan "c" cdeps + cdeps' <- join <$> setBuildPlan "c" pkgSet cdeps let unpacked = fmap (over dirLens (TL.pack d <>)) cdeps' clibs = fmap (buildHelper False) unpacked parallel_ (setup' ++ libs' ++ clibs) diff --git a/src/Language/ATS/Package/Exec.hs b/src/Language/ATS/Package/Exec.hs --- a/src/Language/ATS/Package/Exec.hs +++ b/src/Language/ATS/Package/Exec.hs @@ -18,6 +18,7 @@ import System.Directory import System.IO.Temp (withSystemTempDirectory) +-- TODO command to list available packages. wrapper :: ParserInfo Command wrapper = info (helper <*> versionInfo <*> command') (fullDesc @@ -36,7 +37,7 @@ , _lint :: Bool } | Clean - | Test { _targets :: [String] } + | Test { _targets :: [String], _lint :: Bool } | Fetch { _url :: String } | Nuke | Upgrade @@ -60,7 +61,7 @@ run' = Run <$> targets "run" test' :: Parser Command -test' = Test <$> targets "test" +test' = Test <$> targets "test" <*> noLint valgrind :: Parser Command valgrind = Valgrind <$> targets "run with valgrind" @@ -89,12 +90,15 @@ <> help "Force rebuild of all targets") <*> (length <$> many (flag' () (short 't' <> long "verbose" <> help "Turn up verbosity"))) - <*> fmap not - (switch - (long "no-lint" - <> short 'l' - <> help "Disable the shake linter")) + <*> noLint +noLint :: Parser Bool +noLint = fmap not + (switch + (long "no-lint" + <> short 'l' + <> help "Disable the shake linter")) + fetch :: Parser Command fetch = Fetch <$> argument str @@ -123,12 +127,12 @@ run Upgrade = upgradeAtsPkg run Nuke = cleanAll run (Fetch u) = fetchPkg u -run Clean = mkPkg False False False mempty ["clean"] Nothing 0 +run Clean = mkPkg False False True mempty ["clean"] Nothing 0 run (Build rs rb tgt rba v lint) = runHelper rb rba lint rs tgt v -run c = runHelper False False False rs Nothing 0 +run (Test ts lint) = runHelper False False lint ("test" : ts) Nothing 0 +run c = runHelper False False True rs Nothing 0 where rs = g c g Install = ["install"] - g (Test ts) = "test" : ts g (Valgrind ts) = "valgrind" : ts g (Run ts) = "run" : ts g _ = undefined diff --git a/src/Language/ATS/Package/PackageSet.hs b/src/Language/ATS/Package/PackageSet.hs --- a/src/Language/ATS/Package/PackageSet.hs +++ b/src/Language/ATS/Package/PackageSet.hs @@ -27,16 +27,17 @@ deriving (Interpret, Show) setBuildPlan :: FilePath -- ^ Filepath for cache inside @.atspkg@ + -> String -- ^ URL of package set to use. -> [String] -- ^ Libraries we want -> IO [[ATSDependency]] -setBuildPlan p deps = do +setBuildPlan p url deps = do b <- doesFileExist depCache bool setBuildPlan' (decode <$> BSL.readFile depCache) b where depCache = ".atspkg/buildplan-" ++ p setBuildPlan' = do putStrLn "Resolving dependencies..." - pkgSet <- input auto "https://raw.githubusercontent.com/vmchale/atspkg/master/pkgs/pkg-set.dhall" + pkgSet <- input auto (TL.pack url) case mkBuildPlan pkgSet deps of Right x -> createDirectoryIfMissing True ".atspkg" >> BSL.writeFile depCache (encode x) >> pure x Left x -> resolutionFailed x diff --git a/stack.yaml b/stack.yaml --- a/stack.yaml +++ b/stack.yaml @@ -8,7 +8,7 @@ extra-deps: - shake-ext-2.3.0.0 - composition-prelude-1.1.0.2 - - language-ats-0.2.0.0 + - language-ats-0.3.0.0 - cli-setup-0.2.0.1 - ats-setup-0.3.0.2 - hspec-dirstream-0.3.0.0