nix-thunk 0.2.0.0 → 0.2.0.2
raw patch · 5 files changed
+224/−94 lines, 5 filesdep +cryptonitedep +memorydep +monad-loggerdep −gitdep ~basedep ~cli-extrasdep ~cli-gitnew-uploader
Dependencies added: cryptonite, memory, monad-logger
Dependencies removed: git
Dependency ranges changed: base, cli-extras, cli-git, cli-nix, github, lens, optparse-applicative, time, which
Files
- CHANGELOG.md +6/−0
- README.md +76/−0
- nix-thunk.cabal +71/−60
- src/Nix/Thunk.hs +67/−29
- src/Nix/Thunk/Command.hs +4/−5
CHANGELOG.md view
@@ -1,5 +1,11 @@ # Revision history for nix-thunk +## 0.2.0.2+* Add support for GHC 8.8.4.++## 0.2.0.1+* Fix parsing of --rev arguments+ ## 0.2.0.0 * Add nix-thunk create. This caused some minor breakage to the Haskell library API, but not the Nix or command line interfaces.
+ README.md view
@@ -0,0 +1,76 @@+nix-thunk+=========+[](https://haskell.org) [](https://hackage.haskell.org/package/nix-thunk) [](https://matrix.hackage.haskell.org/#/package/nix-thunk) [](https://github.com/obsidiansystems/nix-thunk/actions) [](https://github.com/obsidiansystems/nix-thunk/blob/master/LICENSE)++nix-thunk is a lightweight Nix dependency manager, focused on making it easy to contribute improvements back to libraries you depend on.++nix-thunk does this by creating and managing "thunks" - directories that stand in for full git repositories. Like git submodules, they pin a specific commit of the target repository, but unlike git submodules, you don't have to clone them to use them. nix-thunk makes them "transparent" to Nix scripts, so any script that calls `import path/to/some/thunk` will work the same on the thunk as it does on the original repository.++* [Installation](#installation)+* [Command Usage](#command-usage)+ * [Create a dependency](#create-a-dependency)+ * [Work on a dependency](#work-on-a-dependency)+ * [Update a dependency](#update-a-dependency)+* [Nix Usage](#nix-usage)+* [Contributing](#contributing)+* [License](#license)++## Installation++```bash+nix-env -f https://github.com/obsidiansystems/nix-thunk/archive/master.tar.gz -iA command+```++## Command Usage++### Create a dependency++```bash+nix-thunk create https://example.com/some-dep+```++If you have already cloned the dependency as a git repository, you can just `pack` it instead:++```bash+nix-thunk pack some-dep+```++### Work on a dependency++If you discover a bug fix or improvement that your dependency needs, you can use `nix-thunk unpack path/to/your/dependency` to turn the thunk back into a full checkout of the repository. Your Nix scripts should continue working, and you can modify the dependency's source code, push it to a branch or a fork, send a pull request, and then use `nix-thunk pack path/to/your/dependency` to pack it back up into a thunk. When the dependency accepts your pull request, you can easily update the thunk.++```bash+nix-thunk unpack some-dep+# Improve some-dep and push your work to a branch+nix-thunk pack some-dep+```++### Update a dependency++For routine updates, you can run `nix-thunk update path/to/your/dependency` to point the thunk at the latest version of the dependency without needing to do a `nix-thunk unpack` or a `git clone`.++```bash+nix-thunk update some-dep+```++## Nix Usage++The [`default.nix`](default.nix) file in this repository also defines the nix function, `thunkSource`. This can be used in your nix files to access the contents of thunks. In the following example, a thunk is used in place of the source location argument to `callCabal2nix`. `thunkSource` works whether the thunk is packed or unpacked, making it convenient to run nix commands with modified thunks.++```nix+ haskellPackages = pkgs.haskell.packages.ghc865.override {+ overrides = self: super: {+ which = self.callCabal2nix "which" (thunkSource ./dep/which) {};+ };+ };+```++## Contributing+Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change. See the [contribution guide](CONTRIBUTING.md) for more details.++## License+[BSD3](./LICENSE)++***++
nix-thunk.cabal view
@@ -1,66 +1,77 @@-cabal-version: >=1.10-name: nix-thunk-version: 0.2.0.0-license: BSD3-license-file: LICENSE-copyright: Obsidian Systems LLC 2020-maintainer: maintainer@obsidian.systems-author: Obsidian Systems LLC-bug-reports: https://github.com/obsidiansystems/nix-thunk-synopsis: Lightweight dependency management with Nix+cabal-version: >=1.10+name: nix-thunk+version: 0.2.0.2+license: BSD3+license-file: LICENSE+copyright: Obsidian Systems LLC 2020+maintainer: maintainer@obsidian.systems+author: Obsidian Systems LLC+bug-reports: https://github.com/obsidiansystems/nix-thunk+synopsis: Lightweight dependency management with Nix description:- nix-thunk lets you manage source code depencies in a lightweight and reproducible way, using Nix. Each source repository is represented by a stub directory, which refers to the original Git repository. nix-thunk can easily update these dependencies.- .- If you need to make improvements to the original repositories, nix-thunk can unpack them in-place, so that changes can be tested right away, and then pack them back up when you're done. This makes it easy to send a pull request to the upstream repo while your project continues on a fork, then switch back to upstream once your pull request has been merged.-category: Nix, Git-build-type: Simple+ nix-thunk lets you manage source code depencies in a lightweight and reproducible way, using Nix. Each source repository is represented by a stub directory, which refers to the original Git repository. nix-thunk can easily update these dependencies.+ .+ If you need to make improvements to the original repositories, nix-thunk can unpack them in-place, so that changes can be tested right away, and then pack them back up when you're done. This makes it easy to send a pull request to the upstream repo while your project continues on a fork, then switch back to upstream once your pull request has been merged.++category: Nix, Git+build-type: Simple extra-source-files:- CHANGELOG.md+ CHANGELOG.md+ README.md +tested-with: GHC ==8.6.5 || ==8.8.4+ library- exposed-modules:- Nix.Thunk- Nix.Thunk.Command- hs-source-dirs: src- default-language: Haskell2010- ghc-options: -Wall- build-depends:- aeson >=1.4.4.0 && <1.5,- aeson-pretty >=0.8.7 && <0.9,- base >=4.12.0.0 && <4.13,- bytestring >=0.10.8.2 && <0.11,- cli-extras >=0.1.0.0 && <0.2,- cli-git >=0.1.0.0 && <0.2,- cli-nix >=0.1.0.0 && <0.2,- containers >=0.6.0.1 && <0.7,- data-default >=0.7.1.1 && <0.8,- directory >=1.3.3.0 && <1.4,- either >=5.0.1.1 && <5.1,- exceptions >=0.10.3 && <0.11,- extra >=1.6.18 && <1.7,- filepath >=1.4.2.1 && <1.5,- git >=0.3.0 && <0.4,- github ==0.22.*,- here >=1.2.13 && <1.3,- lens >=4.17.1 && <4.18,- logging-effect >=1.3.4 && <1.4,- megaparsec >=7.0.5 && <7.1,- modern-uri >=0.3.1.0 && <0.4,- mtl >=2.2.2 && <2.3,- optparse-applicative >=0.14.3.0 && <0.15,- temporary ==1.3.*,- text >=1.2.3.1 && <1.3,- time >=1.8.0.2 && <1.9,- unix >=2.7.2.2 && <2.8,- which >=0.1.0.0 && <0.2,- yaml >=0.11.1.2 && <0.12+ exposed-modules:+ Nix.Thunk+ Nix.Thunk.Command + hs-source-dirs: src+ default-language: Haskell2010+ ghc-options: -Wall+ build-depends:+ aeson >=1.4.4.0 && <1.5+ , aeson-pretty >=0.8.7 && <0.9+ , base >=4.12.0.0 && <4.15+ , bytestring >=0.10.8.2 && <0.11+ , cli-extras >=0.1.0.1 && <0.2+ , cli-git >=0.1.0.1 && <0.2+ , cli-nix >=0.1.0.1 && <0.2+ , containers >=0.6.0.1 && <0.7+ , cryptonite >=0.25 && <0.28+ , data-default >=0.7.1.1 && <0.8+ , directory >=1.3.3.0 && <1.4+ , either >=5.0.1.1 && <5.1+ , exceptions >=0.10.3 && <0.11+ , extra >=1.6.18 && <1.7+ , filepath >=1.4.2.1 && <1.5+ , github >=0.25 && <0.27+ , here >=1.2.13 && <1.3+ , lens >=4.17.1 && <4.20+ , logging-effect >=1.3.4 && <1.4+ , megaparsec >=7.0.5 && <7.1+ , memory >=0.14 && <0.16+ , modern-uri >=0.3.1.0 && <0.4+ , monad-logger >=0.3.30 && <0.4+ , mtl >=2.2.2 && <2.3+ , optparse-applicative >=0.14.3.0 && <0.17+ , temporary >=1.3 && <1.4+ , text >=1.2.3.1 && <1.3+ , time >=1.8.0.2 && <1.12+ , unix >=2.7.2.2 && <2.8+ , which >=0.2 && <0.3+ , yaml >=0.11.1.2 && <0.12+ executable nix-thunk- main-is: src-bin/nix-thunk.hs- default-language: Haskell2010- build-depends:- base >=4.12.0.0 && <4.13,- nix-thunk -any,- cli-extras >=0.1.0.0 && <0.2,- optparse-applicative >=0.14.3.0 && <0.15,- text >=1.2.3.1 && <1.3+ main-is: src-bin/nix-thunk.hs+ default-language: Haskell2010+ build-depends:+ base >=4.12.0.0 && <4.15+ , cli-extras >=0.1.0.1 && <0.2+ , nix-thunk+ , optparse-applicative >=0.14.3.0 && <0.17+ , text >=1.2.3.1 && <1.3++source-repository head+ type: git+ location: git://github.com/obsidiansystems/nix-thunk.git
src/Nix/Thunk.hs view
@@ -1,7 +1,8 @@ {-# LANGUAGE ConstraintKinds #-} {-# LANGUAGE DataKinds #-}+{-# LANGUAGE DeriveDataTypeable #-} {-# LANGUAGE FlexibleContexts #-}-{-# LANGUAGE LambdaCase #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-} {-# LANGUAGE LambdaCase #-} {-# LANGUAGE MultiParamTypeClasses #-} {-# LANGUAGE MultiWayIf #-}@@ -41,30 +42,43 @@ , parseGitUri , GitUri (..) , uriThunkPtr+ , Ref(..)+ , refFromHexString ) where +import Bindings.Cli.Coreutils (cp)+import Bindings.Cli.Git+import Bindings.Cli.Nix+import Cli.Extras import Control.Applicative-import Control.Lens (ifor, ifor_, (.~), makePrisms)+import Control.Exception (Exception, displayException, throw, try)+import Control.Lens ((.~), ifor, ifor_, makePrisms) import Control.Monad+import Control.Monad.Catch (MonadCatch, MonadMask, handle)+import Control.Monad.Except import Control.Monad.Extra (findM)-import Control.Monad.Catch (MonadCatch, handle, MonadMask) import Control.Monad.Fail (MonadFail)-import Control.Monad.Except import Control.Monad.Log (MonadLog)+import Crypto.Hash (Digest, HashAlgorithm, SHA1, digestFromByteString) import Data.Aeson ((.=)) import qualified Data.Aeson as Aeson import Data.Aeson.Encode.Pretty import qualified Data.Aeson.Types as Aeson import Data.Bifunctor (first)+import Data.ByteArray.Encoding (Base(..), convertFromBase, convertToBase)+import Data.ByteString (ByteString)+import qualified Data.ByteString.Char8 as BSC import qualified Data.ByteString.Lazy as LBS import Data.Containers.ListUtils (nubOrd)-import Data.Either.Combinators (fromRight')+import Data.Data (Data)+import Data.Default+import Data.Either.Combinators (fromRight', rightToMaybe) import Data.Foldable (toList)+import Data.Function import Data.Functor ((<&>))-import Data.Git.Ref (Ref)-import qualified Data.Git.Ref as Ref import qualified Data.List as L-import Data.List.NonEmpty (NonEmpty (..), nonEmpty)+import Data.List (stripPrefix)+import Data.List.NonEmpty (NonEmpty(..), nonEmpty) import qualified Data.List.NonEmpty as NonEmpty import Data.Map (Map) import qualified Data.Map as Map@@ -77,27 +91,19 @@ import qualified Data.Text as T import Data.Text.Encoding import qualified Data.Text.IO as T+import Data.Time.Clock.POSIX (posixSecondsToUTCTime)+import Data.Traversable+import Data.Typeable (Typeable) import Data.Yaml (parseMaybe) import GitHub import GitHub.Data.Name import System.Directory import System.Exit import System.FilePath-import System.IO.Temp-import qualified Text.URI as URI-import Cli.Extras-import Control.Exception (displayException, try)-import Data.Either.Combinators (rightToMaybe)-import Data.Traversable import System.IO.Error (isDoesNotExistError)-import Data.Time.Clock.POSIX (posixSecondsToUTCTime)+import System.IO.Temp import System.Posix.Files-import Data.Default-import Data.Function-import Bindings.Cli.Git-import Bindings.Cli.Coreutils (cp)-import Bindings.Cli.Nix-import Data.List (stripPrefix)+import qualified Text.URI as URI -------------------------------------------------------------------------------- -- Hacks@@ -155,7 +161,7 @@ -- | A specific revision of data; it may be available from multiple sources data ThunkRev = ThunkRev- { _thunkRev_commit :: Ref Ref.SHA1+ { _thunkRev_commit :: Ref SHA1 , _thunkRev_nixSha256 :: NixSha256 } deriving (Show, Eq, Ord)@@ -215,7 +221,7 @@ data ThunkCreateConfig = ThunkCreateConfig { _thunkCreateConfig_uri :: GitUri , _thunkCreateConfig_branch :: Maybe (Name Branch)- , _thunkCreateConfig_rev :: Maybe (Ref Ref.SHA1)+ , _thunkCreateConfig_rev :: Maybe (Ref SHA1) , _thunkCreateConfig_config :: ThunkConfig , _thunkCreateConfig_destination :: Maybe FilePath } deriving Show@@ -249,8 +255,8 @@ ThunkSource_GitHub s -> _gitHubSource_branch s ThunkSource_Git s -> _gitSource_branch s -commitNameToRef :: Name Commit -> Ref Ref.SHA1-commitNameToRef (N c) = Ref.fromHex $ encodeUtf8 c+commitNameToRef :: Name Commit -> Ref SHA1+commitNameToRef (N c) = refFromHex $ encodeUtf8 c -- TODO: Use spinner here. getNixSha256ForUriUnpacked@@ -399,7 +405,7 @@ src <- parseSrc v pure $ ThunkPtr { _thunkPtr_rev = ThunkRev- { _thunkRev_commit = Ref.fromHexString rev+ { _thunkRev_commit = refFromHexString rev , _thunkRev_nixSha256 = sha256 } , _thunkPtr_source = src@@ -455,13 +461,13 @@ [ Just $ "owner" .= _gitHubSource_owner s , Just $ "repo" .= _gitHubSource_repo s , ("branch" .=) <$> _gitHubSource_branch s- , Just $ "rev" .= Ref.toHexString (_thunkRev_commit rev)+ , Just $ "rev" .= refToHexString (_thunkRev_commit rev) , Just $ "sha256" .= _thunkRev_nixSha256 rev , Just $ "private" .= _gitHubSource_private s ] ThunkSource_Git s -> encodePretty' plainGitCfg $ Aeson.object $ catMaybes [ Just $ "url" .= gitUriToText (_gitSource_url s)- , Just $ "rev" .= Ref.toHexString (_thunkRev_commit rev)+ , Just $ "rev" .= refToHexString (_thunkRev_commit rev) , ("branch" .=) <$> _gitSource_branch s , Just $ "sha256" .= _thunkRev_nixSha256 rev , Just $ "fetchSubmodules" .= _gitSource_fetchSubmodules s@@ -924,7 +930,7 @@ ++ [ T.unpack $ gitUriToText $ _gitSource_url gitSrc ] ++ do branch <- maybeToList $ _gitSource_branch gitSrc [ "--branch", T.unpack $ untagName branch ]- git ["reset", "--hard", Ref.toHexString commit]+ git ["reset", "--hard", refToHexString commit] when (_gitSource_fetchSubmodules gitSrc) $ git ["submodule", "update", "--recursive", "--init"] @@ -1298,3 +1304,35 @@ guard $ isNothing (T.findIndex (=='/') authAndHostname) && not (T.null colonAndPath) URI.mkURI properUri++-- The following code has been adapted from the 'Data.Git.Ref',+-- which is apparently no longer maintained++-- | Represent a git reference (SHA1)+newtype Ref hash = Ref { unRef :: Digest hash }+ deriving (Eq, Ord, Typeable)++-- | Invalid Reference exception raised when+-- using something that is not a ref as a ref.+newtype RefInvalid = RefInvalid { unRefInvalid :: ByteString }+ deriving (Show, Eq, Data, Typeable)++instance Exception RefInvalid++refFromHexString :: HashAlgorithm hash => String -> Ref hash+refFromHexString = refFromHex . BSC.pack++refFromHex :: HashAlgorithm hash => BSC.ByteString -> Ref hash+refFromHex s =+ case convertFromBase Base16 s :: Either String ByteString of+ Left _ -> throw $ RefInvalid s+ Right h -> case digestFromByteString h of+ Nothing -> throw $ RefInvalid s+ Just d -> Ref d++-- | transform a ref into an hexadecimal string+refToHexString :: Ref hash -> String+refToHexString (Ref d) = show d++instance Show (Ref hash) where+ show (Ref bs) = BSC.unpack $ convertToBase Base16 bs
src/Nix/Thunk/Command.hs view
@@ -2,18 +2,17 @@ {-# LANGUAGE LambdaCase #-} module Nix.Thunk.Command where +import Cli.Extras (HasCliConfig, Output) import Control.Monad.Catch (MonadMask) import Control.Monad.Error.Class (MonadError) import Control.Monad.Fail (MonadFail) import Control.Monad.IO.Class (MonadIO) import Control.Monad.Log (MonadLog)-import Data.List.NonEmpty (NonEmpty (..))+import Data.List.NonEmpty (NonEmpty(..))+import qualified Data.Text as T import Nix.Thunk-import Cli.Extras (HasCliConfig, Output) import Options.Applicative import System.FilePath-import Data.Git.Ref-import qualified Data.Text as T thunkConfig :: Parser ThunkConfig thunkConfig = ThunkConfig@@ -37,7 +36,7 @@ thunkCreateConfig = ThunkCreateConfig <$> argument (maybeReader (parseGitUri . T.pack)) (metavar "URI" <> help "Address of the target repository") <*> optional (strOption (short 'b' <> long "branch" <> metavar "BRANCH" <> help "Point the new thunk at the given branch"))- <*> optional (option (fromHexString <$> auto) (long "rev" <> long "revision" <> metavar "REVISION" <> help "Point the new thunk at the given revision"))+ <*> optional (option (refFromHexString <$> str) (long "rev" <> long "revision" <> metavar "REVISION" <> help "Point the new thunk at the given revision")) <*> thunkConfig <*> optional (strArgument (action "directory" <> metavar "DESTINATION" <> help "The name of a new directory to create for the thunk"))