nvfetcher 0.1.0.0 → 0.2.0.0
raw patch · 14 files changed
+382/−197 lines, 14 filesdep +containersdep +microlensdep +microlens-thdep −unordered-containers
Dependencies added: containers, microlens, microlens-th, optparse-simple
Dependencies removed: unordered-containers
Files
- CHANGELOG.md +7/−0
- Main_example.hs +1/−1
- README.md +57/−22
- app/Config.hs +53/−16
- app/Main.hs +94/−37
- nvfetcher.cabal +12/−8
- src/NvFetcher.hs +19/−41
- src/NvFetcher/Core.hs +7/−7
- src/NvFetcher/NixFetcher.hs +17/−17
- src/NvFetcher/Nvchecker.hs +9/−9
- src/NvFetcher/PackageSet.hs +63/−14
- src/NvFetcher/ShakeExtras.hs +7/−7
- src/NvFetcher/Types.hs +18/−18
- src/NvFetcher/Types/Lens.hs +18/−0
CHANGELOG.md view
@@ -1,5 +1,12 @@ # Revision history for nvfetcher +## 0.2.0.0++* Generated package sources will be sorted alphabetically.+* CLI program now supports `nix-git-prefetch` configurations in TOML.+* Lenses are added for some data types.+* CLI options are no loger inherited from Shake. Now `nvfetcher` has its own CLI options with completion support.+ ## 0.1.0.0 * First version. Released on an unsuspecting world.
Main_example.hs view
@@ -11,7 +11,7 @@ packageSet = do define $ package "feeluown-core" `fromPypi` "feeluown" - define $ package "qliveplayer" `fromGitHub` ("IsoaSFlus", "QLivePlayer")+ define $ package "qliveplayer" `fromGitHub'` ("IsoaSFlus", "QLivePlayer", fetchSubmodules .~ True) define $ package "fcitx5-pinyin-zhwiki"
README.md view
@@ -23,6 +23,7 @@ [qliveplayer] src.github = "IsoaSFlus/QLivePlayer" fetch.github = "IsoaSFlus/QLivePlayer"+git.fetchSubmodules = true ``` running `nvfetcher build` will create `sources.nix` like:@@ -33,22 +34,22 @@ { feeluown-core = { pname = "feeluown-core";- version = "3.7.6";+ version = "3.7.7"; src = fetchurl {- sha256 = "1bsz149dv3j5sfbynjrqsqbkkxdxkdlq4sdx2vi8whvfwfg0j2f0";- url = "https://pypi.io/packages/source/f/feeluown/feeluown-3.7.6.tar.gz";+ sha256 = "06d3j39ff9znqxkhp9ly81lcgajkhg30hyqxy2809yn23xixg3x2";+ url = "https://pypi.io/packages/source/f/feeluown/feeluown-3.7.7.tar.gz"; }; }; qliveplayer = { pname = "qliveplayer";- version = "3.22.0";+ version = "3.22.1"; src = fetchgit { url = "https://github.com/IsoaSFlus/QLivePlayer";- rev = "3.22.0";- fetchSubmodules = false;+ rev = "3.22.1";+ fetchSubmodules = true; deepClone = false; leaveDotGit = false;- sha256 = "192g42pvibms2rsjh68fck4bj59b10ay9zqcf2hqhcka0xmnyb09";+ sha256 = "00zqg28q5xrbgql0kclgkhd15fc02qzsrvi0qg8lg3qf8a53v263"; }; }; }@@ -58,7 +59,7 @@ and nvfetcher will help us keep their version and prefetched SHA256 sums up-to-date, stored in `sources.nix`. Shake will help us handle necessary rebuilds -- we check versions of packages during each run, but only prefetch them when needed. -## Live examples+### Live examples How to use the generated sources file? Here are some examples: @@ -66,30 +67,47 @@ * Nick Cao's [flakes repo](https://gitlab.com/NickCao/flakes/-/tree/master/pkgs) -## Usage+## Installation -Basically, there are two ways to use nvfetcher, where the difference is how we provide package sources definitions to it. -No matter which way you use it in, CLI options are inherited from shake with two targets, typically used as:+`nvfetcher` package is available in [nixpkgs](https://github.com/NixOS/nixpkgs), so you can try it with: -* `nvfetcher build` - our main purpose, creating `sources.nix`-* `nvfetcher clean` - clean up cache and remove `sources.nix`+```+$ nix-shell -p nvfetcher+``` -> nvfetcher uses `build` as the target if no specified+This repo also has flakes support: -> You can specify `-j` to enable parallelism+```+$ nix run github:berberman/nvfetcher+``` +To use it as a Haskell library, the package is available on [Hackage](https://hackage.haskell.org/package/nvfetcher).++## Usage++Basically, there are two ways to use nvfetcher, where the difference is how we provide package sources definitions to it.+ ### CLI To run nvfetcher as a CLI program, you'll need to provide package sources defined in TOML. -Aavailable CLI options:-* `-c` (`--config`) - path to the TOML configuration file-* `-o` (`--output`) - path to the output nix file-* `-v` (`--version`) - print nvfetcher version-* `-l` (`--log`) - path to log file, where nvfetcher dumps the version changes +```+Available options:+ --version Show version+ --help Show this help text+ -c,--config FILE Path to nvfetcher TOML config+ (default: "nvfetcher.toml")+ -o,--output FILE Path to output nix file (default: "sources.nix")+ -l,--changelog FILE Dump version changes to a file+ -j NUM Number of threads (0: detected number of processors)+ (default: 0)+ -t,--timing Show build time+ -v,--verbose Verbose mode+ TARGET Two targets are available: build and clean+``` Each *package* corresponds to a TOML table, whose name is encoded as table key;-there are two pairs in each table:+there are two fields and four optional git prefetch configuration in each table: * a nvchecker configuration, how to track version updates * `src.github = owner/repo` - the latest gituhb release * `src.pypi = pypi_name` - the latest pypi release@@ -104,11 +122,20 @@ * `fetch.git = git_url` or `git_url:rev` (default to `$ver` if no `rev` specified) * `fetch.url = url` +* optional git prefetch configuration, which makes sense only when the fetcher equals to `fetch.github` or `fetch.git`.+They can exist simultanesouly.+ * `git.branch = branch_name` - branch to fetch+ * `git.deepClone` - a bool value to control deep clone+ * `git.fetchSubmodules` - a bool value to control fetching submodules+ * `git.leaveDotGit` - a bool value to control leaving dot git+ You can find an example of the configuration file, see [`nvfetcher_example.toml`](nvfetcher_example.toml). ### Haskell library -nvfetcher itsetlf is a Haskell library as well, whereas the CLI program is just a trivial wrapper of the library. You can create a Haskell program depending on it directly, creating an entry point. In this case, we can define packages in Haskell language, getting rid of TOML constraints.+nvfetcher itsetlf is a Haskell library as well, whereas the CLI program is just a trivial wrapper of the library.+You can create a Haskell program depending on it directly, by using the `runNvFetcher` entry point.+In this case, we can define packages in Haskell language, getting rid of TOML constraints. You can find an example of using nvfetcher in the library way, see [`Main_example.hs`](Main_example.hs). @@ -130,3 +157,11 @@ ## Contributing Issues and PRs are always welcome. **\_(:з」∠)\_**++Building from source:++```+$ git clone https://github.com/berberman/nvfetcher+$ nix develop+$ cabal build+```
app/Config.hs view
@@ -7,10 +7,15 @@ module Config where import Control.Applicative ((<|>))+import Data.Coerce (coerce) import Data.List.NonEmpty (NonEmpty ((:|)))+import Data.Maybe (fromMaybe) import qualified Data.Text as T-import NvFetcher+import Lens.Micro+import Lens.Micro.Extras (view) import NvFetcher.NixFetcher+import NvFetcher.Types+import NvFetcher.Types.Lens import Toml (TOML, TomlCodec, (.=)) import qualified Toml import Validation (validationToEither)@@ -22,8 +27,21 @@ go (Right x : xs) se sp = go xs se (x : sp) go [] [] sp = Right sp go [] se _ = Left se- tables = [fmap (uncurry (Package k)) $ validationToEither $ Toml.runTomlCodec iCodec v | (Toml.unKey -> (Toml.unPiece -> k) :| _, v) <- Toml.toList $ Toml.tomlTables toml]- iCodec = (,) <$> versionSourceCodec .= fst <*> fetcherCodec .= snd+ tables = [fmap (toPackage k) $ validationToEither $ Toml.runTomlCodec iCodec v | (Toml.unKey -> (Toml.unPiece -> k) :| _, v) <- Toml.toList $ Toml.tomlTables toml]+ toPackage k (v, f, g@GitOptions {..}) =+ let f' v = case f v of+ x@FetchGit {} ->+ x+ & branch .~ goBranch+ & deepClone .~ fromMaybe False goDeepClone+ & fetchSubmodules .~ fromMaybe False goFetchSubmodules+ & leaveDotGit .~ fromMaybe False goLeaveDotGit+ x ->+ if not $ isGitOptionsDefault g+ then error $ "Try to set git-prefetch configuration for a url fetcher: " <> show x+ else x+ in Package k v f'+ iCodec = (,,) <$> versionSourceCodec .= (view _1) <*> fetcherCodec .= (view _2) <*> gitOptionsCodec .= (view _3) versionSourceCodec :: TomlCodec VersionSource versionSourceCodec =@@ -35,46 +53,46 @@ id ( Toml.textBy ( \case- GitHubRelease {..} -> owner <> "/" <> repo+ GitHubRelease {..} -> _owner <> "/" <> _repo _ -> error "impossible" ) ( \x -> case T.split (== '/') x of- [owner, repo] -> Right GitHubRelease {..}+ [_owner, _repo] -> Right GitHubRelease {..} _ -> Left "unexpected github srouce: it should be something like [owner]/[repo]" ) "src.github" ) <|> Toml.dimatch ( \case- Git {..} -> Just vurl+ Git {..} -> Just _vurl _ -> Nothing ) Git (Toml.text "src.git") <|> Toml.dimatch ( \case- Pypi {..} -> Just pypi+ Pypi {..} -> Just _pypi _ -> Nothing ) Pypi (Toml.text "src.pypi") <|> Toml.dimatch ( \case- ArchLinux {..} -> Just archpkg+ ArchLinux {..} -> Just _archpkg _ -> Nothing ) ArchLinux (Toml.text "src.archpkg") <|> Toml.dimatch ( \case- Aur {..} -> Just aur+ Aur {..} -> Just _aur _ -> Nothing ) Aur (Toml.text "src.aur") <|> Toml.dimatch ( \case- Manual {..} -> Just manual+ Manual {..} -> Just _manual _ -> Nothing ) Manual@@ -87,18 +105,18 @@ id ( Toml.textBy ( \case- Repology {..} -> repology <> ":" <> repo+ Repology {..} -> _repology <> ":" <> _repo _ -> error "impossible" ) ( \t -> case T.split (== ':') t of- [repology, repo] -> Right Repology {..}+ [_repology, _repo] -> Right Repology {..} _ -> Left "unexpected repology source: it should be something like [project]:[repo]" ) "src.repology" ) unsupportError :: a-unsupportError = error "serialization of fetchers is unsupported"+unsupportError = error "serialization is unsupported" -- | Use it only for deserialization!!! fetcherCodec :: TomlCodec PackageFetcher@@ -108,7 +126,7 @@ ( \t -> case T.split (== '/') t of [owner, rest] -> case T.split (== ':') rest of [repo, rawV] ->- Right $ \(coerce -> realV) -> gitHubFetcher (owner, repo) $ coerce $ T.replace "$ver" rawV realV+ Right $ \(coerce -> realV) -> gitHubFetcher (owner, repo) $ coerce $ T.replace "$ver" realV rawV [repo] -> Right $ gitHubFetcher (owner, repo) _ -> Left "unexpected github fetcher: it should be something like [owner]/[repo] or [owner]/[repo]:[ver]" _ -> Left "unexpected github fetcher: it should be something like [owner]/[repo] or [owner]/[repo]:[ver]"@@ -118,7 +136,7 @@ unsupportError ( \t -> case T.split (== ':') t of [fpypi, rawV] ->- Right $ \(coerce -> realV) -> pypiFetcher fpypi $ coerce $ T.replace "$ver" rawV realV+ Right $ \(coerce -> realV) -> pypiFetcher fpypi $ coerce $ T.replace "$ver" realV rawV [fpypi] -> Right $ pypiFetcher fpypi _ -> Left "unexpected pypi fetcher: it should be something like [pypi] or [pypi]:[ver]" )@@ -127,7 +145,7 @@ unsupportError ( \t -> case T.split (== ':') t of [furl, rawV] ->- Right $ \(coerce -> realV) -> gitFetcher furl $ coerce $ T.replace "$ver" rawV realV+ Right $ \(coerce -> realV) -> gitFetcher furl $ coerce $ T.replace "$ver" realV rawV [furl] -> Right $ gitFetcher furl _ -> Left "unexpected git fetcher: it should be something like [git_url] or [git_url]:[ver]" )@@ -136,3 +154,22 @@ unsupportError (\t -> Right $ \(coerce -> v) -> urlFetcher $ T.replace "$ver" v t) "fetch.url"++data GitOptions = GitOptions+ { goBranch :: Maybe T.Text,+ goDeepClone :: Maybe Bool,+ goFetchSubmodules :: Maybe Bool,+ goLeaveDotGit :: Maybe Bool+ }+ deriving (Eq)++isGitOptionsDefault :: GitOptions -> Bool+isGitOptionsDefault = (== GitOptions Nothing Nothing Nothing Nothing)++gitOptionsCodec :: TomlCodec GitOptions+gitOptionsCodec =+ GitOptions+ <$> Toml.dioptional (Toml.text "git.branch") .= goBranch+ <*> Toml.dioptional (Toml.bool "git.deepClone") .= goDeepClone+ <*> Toml.dioptional (Toml.bool "git.fetchSubmodules") .= goFetchSubmodules+ <*> Toml.dioptional (Toml.bool "git.leaveDotGit") .= goLeaveDotGit
app/Main.hs view
@@ -1,5 +1,6 @@ {-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE RecordWildCards #-}+{-# LANGUAGE TemplateHaskell #-} {-# LANGUAGE ViewPatterns #-} module Main where@@ -7,56 +8,112 @@ import Config import qualified Data.Text as T import qualified Data.Text.IO as T-import Data.Version (showVersion) import Development.Shake import NvFetcher-import Paths_nvfetcher-import System.Console.GetOpt+import Options.Applicative.Simple+import qualified Paths_nvfetcher as Paths import qualified Toml data CLIOptions = CLIOptions { configPath :: FilePath, outputPath :: FilePath,- versionMode :: Bool,- logPath :: Maybe FilePath+ logPath :: Maybe FilePath,+ threads :: Int,+ timing :: Bool,+ verbose :: Bool,+ target :: String } deriving (Show) -defaultCLIArgs :: CLIOptions-defaultCLIArgs = CLIOptions "nvfetcher.toml" "sources.nix" False Nothing+cliOptionsParser :: Parser CLIOptions+cliOptionsParser =+ CLIOptions+ <$> strOption+ ( long "config"+ <> short 'c'+ <> metavar "FILE"+ <> help "Path to nvfetcher TOML config"+ <> value "nvfetcher.toml"+ <> showDefault+ <> completer (bashCompleter "file")+ )+ <*> strOption+ ( long "output"+ <> short 'o'+ <> metavar "FILE"+ <> help "Path to output nix file"+ <> showDefault+ <> value "sources.nix"+ <> completer (bashCompleter "file")+ )+ <*> optional+ ( strOption+ ( long "changelog"+ <> short 'l'+ <> metavar "FILE"+ <> help "Dump version changes to a file"+ <> completer (bashCompleter "file")+ )+ )+ <*> ( option+ auto+ ( short 'j'+ <> metavar "NUM"+ <> help "Number of threads (0: detected number of processors)"+ <> value 0+ <> showDefault+ )+ )+ <*> switch (long "timing" <> short 't' <> help "Show build time")+ <*> switch (long "verbose" <> short 'v' <> help "Verbose mode")+ <*> strArgument+ ( metavar "TARGET"+ <> help "Two targets are available: build and clean"+ <> value "build"+ <> completer (listCompleter ["build", "clean"])+ ) -flags :: [OptDescr (Either String (CLIOptions -> CLIOptions))]-flags =- [ Option ['c'] ["config"] (ReqArg (\s -> Right $ \o -> o {configPath = s}) "FILE") "Path to nvfetcher config",- Option ['o'] ["output"] (ReqArg (\s -> Right $ \o -> o {outputPath = s}) "FILE") "Path to output nix file",- Option ['v'] ["version"] (NoArg $ Right $ \o -> o {versionMode = True}) "Print nvfetcher version",- Option ['l'] ["log"] (ReqArg (\s -> Right $ \o -> o {logPath = Just s}) "FILE") "Path to log file"- ]+version :: String+version = $(simpleVersion Paths.version) +getCLIOptions :: IO CLIOptions+getCLIOptions = do+ (opts, ()) <-+ simpleOptions+ version+ "nvfetcher - generate nix sources expr for the latest version of packages"+ ( unlines+ [ "It's important to keep .shake dir to make caches work properly.",+ "If you change the version source or fetcher of an existing package, please run target \"clean\" to rebuild everything."+ ]+ )+ cliOptionsParser+ empty+ pure opts++main :: IO ()+main = do+ CLIOptions {..} <- getCLIOptions+ let args =+ defaultArgs+ { argOutputFilePath = outputPath,+ argActionAfterBuild = maybe (pure ()) logChangesToFile logPath,+ argTarget = target,+ argShakeOptions =+ (argShakeOptions defaultArgs)+ { shakeTimings = timing,+ shakeVerbosity = if verbose then Verbose else Info,+ shakeThreads = threads+ }+ }+ toml <- Toml.parse <$> T.readFile configPath+ case toml of+ Left e -> error $ T.unpack $ Toml.prettyTomlDecodeError $ Toml.ParseError e+ Right x -> case parseConfig x of+ Left e -> error $ T.unpack $ Toml.prettyTomlDecodeErrors e+ Right pkgs -> runNvFetcher args $ purePackageSet pkgs+ logChangesToFile :: FilePath -> Action () logChangesToFile fp = do changes <- getVersionChanges writeFile' fp $ unlines $ show <$> changes--main :: IO ()-main = runNvFetcherWith flags $ \flagValues -> do- let CLIOptions {..} = foldl (flip id) defaultCLIArgs flagValues- if versionMode- then putStrLn ("nvfetcher " <> showVersion version) >> pure Nothing- else do- toml <- Toml.parse <$> T.readFile configPath- case toml of- Left e -> error $ T.unpack $ Toml.prettyTomlDecodeError $ Toml.ParseError e- Right x -> case parseConfig x of- Left e -> error $ T.unpack $ Toml.prettyTomlDecodeErrors e- Right x ->- pure $- Just- ( defaultArgs- { argOutputFilePath = outputPath,- argActionAfterBuild = case logPath of- Just fp -> logChangesToFile fp- _ -> pure ()- },- purePackageSet x- )
nvfetcher.cabal view
@@ -1,6 +1,6 @@ cabal-version: 2.4 name: nvfetcher-version: 0.1.0.0+version: 0.2.0.0 synopsis: Generate nix sources expr for the latest version of packages @@ -24,17 +24,19 @@ common common-options build-depends:- , aeson ^>=1.5.6- , base >=4.8 && <5+ , aeson ^>=1.5.6+ , base >=4.8 && <5 , binary , bytestring- , extra ^>=1.7.9- , free ^>=5.1.5- , neat-interpolation ^>=0.5.1- , shake ^>=0.19.4+ , containers+ , extra ^>=1.7.9+ , free ^>=5.1.5+ , microlens+ , microlens-th+ , neat-interpolation ^>=0.5.1+ , shake ^>=0.19.4 , text , transformers- , unordered-containers ghc-options: -Wall -Wcompat -Widentities -Wincomplete-uni-patterns@@ -56,6 +58,7 @@ NvFetcher.PackageSet NvFetcher.ShakeExtras NvFetcher.Types+ NvFetcher.Types.Lens executable nvfetcher import: common-options@@ -70,6 +73,7 @@ , nvfetcher , tomland ^>=1.3.2 , validation-selective+ , optparse-simple ghc-options: -threaded -rtsopts -with-rtsopts=-N
src/NvFetcher.hs view
@@ -20,7 +20,7 @@ -- import NvFetcher -- -- main :: IO ()--- main = defaultMain defaultArgs packageSet+-- main = runNvFetcher defaultArgs packageSet -- -- packageSet :: PackageSet () -- packageSet = do@@ -42,14 +42,12 @@ ( Args (..), defaultArgs, runNvFetcher,- runNvFetcherWith, module NvFetcher.PackageSet, module NvFetcher.Types, module NvFetcher.ShakeExtras, ) where -import Control.Monad.Trans.Maybe import Data.Text (Text) import qualified Data.Text as T import Development.Shake@@ -60,12 +58,13 @@ import NvFetcher.PackageSet import NvFetcher.ShakeExtras import NvFetcher.Types-import System.Console.GetOpt (OptDescr) -- | Arguments for running nvfetcher data Args = Args- { -- | tweak shake options- argShakeOptions :: ShakeOptions -> ShakeOptions,+ { -- | Shake options+ argShakeOptions :: ShakeOptions,+ -- | Build target+ argTarget :: String, -- | Output file path argOutputFilePath :: FilePath, -- | Custom rules@@ -82,12 +81,11 @@ defaultArgs :: Args defaultArgs = Args- ( \x ->- x- { shakeTimings = True,- shakeProgress = progressSimple- }+ ( shakeOptions+ { shakeProgress = progressSimple+ } )+ "build" "sources.nix" (pure ()) (pure ())@@ -95,38 +93,18 @@ -- | Entry point of nvfetcher runNvFetcher :: Args -> PackageSet () -> IO ()-runNvFetcher args packageSet = runNvFetcherWith [] $ const $ pure $ Just (args, packageSet)---- | Like 'runNvFetcher' but allows to define custom cli flags-runNvFetcherWith ::- -- | Custom flags- [OptDescr (Either String a)] ->- -- | Continuation, the build system won't run if it returns Nothing- ([a] -> IO (Maybe (Args, PackageSet ()))) ->- IO ()-runNvFetcherWith flags f = do- shakeArgsOptionsWith- shakeOptions- flags- $ \opts flagValues argValues -> runMaybeT $ do- (args@Args {..}, packageSet) <- MaybeT $ f flagValues- pkgs <- liftIO $ runPackageSet packageSet- shakeExtras <- liftIO $ initShakeExtras pkgs- let opts' =- let old = argShakeOptions opts- in old- { shakeExtra = addShakeExtra shakeExtras (shakeExtra old)- }- rules = mainRules args- pure $ case argValues of- [] -> (opts', want ["build"] >> rules)- files -> (opts', want files >> rules)+runNvFetcher args@Args {..} packageSet = do+ pkgs <- runPackageSet packageSet+ shakeExtras <- initShakeExtras pkgs+ let opts =+ argShakeOptions+ { shakeExtra = addShakeExtra shakeExtras (shakeExtra argShakeOptions)+ }+ rules = mainRules args+ shake opts $ want [argTarget] >> rules mainRules :: Args -> Rules () mainRules Args {..} = do- addHelpSuffix "It's important to keep .shake dir if you want to get correct version changes and proper cache"- addHelpSuffix "Changing input packages will lead to a fully cleanup, requiring to rebuild everything next run"- "clean" ~> do removeFilesAfter ".shake" ["//*"] removeFilesAfter "." [argOutputFilePath]@@ -141,7 +119,7 @@ else do putInfo "Changes:" putInfo $ unlines $ show <$> changes- writeFileChanged argOutputFilePath $ T.unpack $ srouces $ T.unlines body+ writeFileChanged argOutputFilePath $ T.unpack $ srouces (T.unlines body) <> "\n" putInfo $ "Generate " <> argOutputFilePath argActionAfterBuild
src/NvFetcher/Core.hs view
@@ -42,23 +42,23 @@ lookupPackage pkg >>= \case Nothing -> fail $ "Unkown package key: " <> show pkg Just Package {..} -> do- (NvcheckerResult version mOldV) <- checkVersion pversion pkg- prefetched <- prefetch $ pfetcher version+ (NvcheckerResult version mOldV) <- checkVersion _pversion pkg+ prefetched <- prefetch $ _pfetcher version case mOldV of Nothing ->- recordVersionChange pname Nothing version+ recordVersionChange _pname Nothing version Just old | old /= version ->- recordVersionChange pname (Just old) version+ recordVersionChange _pname (Just old) version _ -> pure ()- let result = gen pname version prefetched+ let result = gen _pname version prefetched pure $ RunResult ChangedRecomputeDiff (encode' (result, version)) result -- | Run the core rule. -- Given a package key, run nvchecker and then prefetch it, -- resulting a nix source snippet like: ----- @@+-- @ -- feeluown-core = { -- pname = "feeluown-core"; -- version = "3.7.7";@@ -67,7 +67,7 @@ -- url = "https://pypi.io/packages/source/f/feeluown/feeluown-3.7.7.tar.gz"; -- }; -- };--- @@+-- @ generateNixSourceExpr :: PackageKey -> Action NixExpr generateNixSourceExpr k = apply1 $ WithPackageKey (Core, k)
src/NvFetcher/NixFetcher.hs view
@@ -66,7 +66,7 @@ instance ToNixExpr (NixFetcher Prefetched) where -- add quotation marks- toNixExpr f = buildNixFetcher (T.pack $ show $ T.unpack $ coerce $ sha256 f) f+ toNixExpr f = buildNixFetcher (T.pack $ show $ T.unpack $ coerce $ _sha256 f) f instance ToNixExpr Bool where toNixExpr True = "true"@@ -81,20 +81,20 @@ let parser = A.withObject "nix-prefetch-git" $ \o -> SHA256 <$> o A..: "sha256" (CmdTime t, Stdout out, CmdLine c) <- command [EchoStderr False] "nix-prefetch-git" $- [T.unpack furl]- <> ["--rev", T.unpack $ coerce rev]- <> ["--fetch-submodules" | fetchSubmodules]- <> ["--branch-name " <> T.unpack b | b <- maybeToList branch]- <> ["--deepClone" | deepClone]- <> ["--leave-dotGit" | leaveDotGit]- putInfo $ "Finishing running " <> c <> ", took " <> show t <> "s"+ [T.unpack _furl]+ <> ["--rev", T.unpack $ coerce _rev]+ <> ["--fetch-submodules" | _fetchSubmodules]+ <> concat [["--branch-name", T.unpack b] | b <- maybeToList _branch]+ <> ["--deepClone" | _deepClone]+ <> ["--leave-dotGit" | _leaveDotGit]+ putVerbose $ "Finishing running " <> c <> ", took " <> show t <> "s" let result = A.parseMaybe parser <=< A.decodeStrict $ out case result of Just x -> pure x _ -> fail $ "Failed to parse output from nix-prefetch-git: " <> T.unpack (T.decodeUtf8 out) FetchUrl {..} -> do- (CmdTime t, Stdout (T.decodeUtf8 -> out), CmdLine c) <- command [EchoStderr False] "nix-prefetch-url" [T.unpack furl]- putInfo $ "Finishing running " <> c <> ", took " <> show t <> "s"+ (CmdTime t, Stdout (T.decodeUtf8 -> out), CmdLine c) <- command [EchoStderr False] "nix-prefetch-url" [T.unpack _furl]+ putVerbose $ "Finishing running " <> c <> ", took " <> show t <> "s" case takeWhile (not . T.null) $ reverse $ T.lines out of [x] -> pure $ coerce x _ -> fail $ "Failed to parse output from nix-prefetch-url: " <> T.unpack out@@ -102,16 +102,16 @@ buildNixFetcher :: Text -> NixFetcher k -> Text buildNixFetcher sha256 = \case FetchGit- { sha256 = _,- rev = toNixExpr -> rev,- fetchSubmodules = toNixExpr -> fetchSubmodules,- deepClone = toNixExpr -> deepClone,- leaveDotGit = toNixExpr -> leaveDotGit,+ { _sha256 = _,+ _rev = toNixExpr -> rev,+ _fetchSubmodules = toNixExpr -> fetchSubmodules,+ _deepClone = toNixExpr -> deepClone,+ _leaveDotGit = toNixExpr -> leaveDotGit, .. } -> [trimming| fetchgit {- url = "$furl";+ url = "$_furl"; rev = "$rev"; fetchSubmodules = $fetchSubmodules; deepClone = $deepClone;@@ -139,7 +139,7 @@ prefetchRule = void $ addOracleCache $ \(f :: NixFetcher Fresh) -> do sha256 <- runFetcher f- pure $ f {sha256 = sha256}+ pure $ f {_sha256 = sha256} -- | Run nix fetcher prefetch :: NixFetcher Fresh -> Action (NixFetcher Prefetched)
src/NvFetcher/Nvchecker.hs view
@@ -58,7 +58,7 @@ writeFile' config $ T.unpack $ genNvConfig "pkg" q need [config] (CmdTime t, Stdout out, CmdLine c) <- cmd $ "nvchecker --logger json -c " <> config- putInfo $ "Finishing running " <> c <> ", took " <> show t <> "s"+ putVerbose $ "Finishing running " <> c <> ", took " <> show t <> "s" let out' = T.decodeUtf8 out result = mapMaybe (A.decodeStrict . T.encodeUtf8) (T.lines out') now <- case result of@@ -79,48 +79,48 @@ [trimming| [$srcName] source = "github"- github = "$owner/$repo"+ github = "$_owner/$_repo" use_latest_release = true |] Git {..} -> [trimming| [$srcName] source = "git"- git = "$vurl"+ git = "$_vurl" use_commit = true |] Aur {..} -> [trimming| [$srcName] source = "aur"- aur = "$aur"+ aur = "$_aur" strip_release = true |] ArchLinux {..} -> [trimming| [$srcName] source = "archpkg"- archpkg = "$archpkg"+ archpkg = "$_archpkg" strip_release = true |] Pypi {..} -> [trimming| [$srcName] source = "pypi"- pypi = "$pypi"+ pypi = "$_pypi" |] Manual {..} -> [trimming| [$srcName] source = "manual"- manual = "$manual"+ manual = "$_manual" |] Repology {..} -> [trimming| [$srcName] source = "repology"- repology = "$repology"- repo = "$repo"+ repology = "$_repology"+ repo = "$_repo" |] -- | Run nvchecker
src/NvFetcher/PackageSet.hs view
@@ -41,6 +41,7 @@ -- ** Two-in-one functions fromGitHub,+ fromGitHub', fromPypi, -- ** Version sources@@ -54,9 +55,11 @@ -- ** Fetchers fetchGitHub,+ fetchGitHub', fetchGitHubRelease, fetchPypi, fetchGit,+ fetchGit', fetchUrl, -- ** Miscellaneous@@ -65,6 +68,13 @@ NotElem, coerce, liftIO,++ -- * Lenses+ (&),+ (.~),+ (%~),+ (^.),+ module NvFetcher.Types.Lens, ) where @@ -72,12 +82,14 @@ import Control.Monad.IO.Class import Data.Coerce (coerce) import Data.Kind (Constraint, Type)-import Data.HashMap.Strict as HMap+import Data.Map.Strict as HMap import Data.Maybe (isJust) import Data.Text (Text) import GHC.TypeLits+import Lens.Micro import NvFetcher.NixFetcher import NvFetcher.Types+import NvFetcher.Types.Lens -------------------------------------------------------------------------------- @@ -118,13 +130,13 @@ -- -- Throws exception as more then one packages with the same name -- are defined-runPackageSet :: PackageSet () -> IO (HashMap PackageKey Package)+runPackageSet :: PackageSet () -> IO (Map PackageKey Package) runPackageSet = \case Free (NewPackage p g) -> runPackageSet g >>= \m ->- if isJust (HMap.lookup (PackageKey $ pname p) m)- then fail $ "Duplicate package name: " <> show (pname p)- else pure $ HMap.insert (PackageKey $ pname p) p m+ if isJust (HMap.lookup (PackageKey $ _pname p) m)+ then fail $ "Duplicate package name: " <> show (_pname p)+ else pure $ HMap.insert (PackageKey $ _pname p) p m Free (EmbedIO action g) -> action >>= runPackageSet . g Pure _ -> pure mempty @@ -179,7 +191,7 @@ -- Example: -- -- @--- define $ package "nvfetcher-git" `sourceGit` "nvfetcher" `fetchGitHub` ("berberman", "nvfetcher")+-- define $ package "nvfetcher-git" `sourceGit` "https://github.com/berberman/nvfetcher" `fetchGitHub` ("berberman", "nvfetcher") -- @ define :: ( Member PackageName r,@@ -213,8 +225,16 @@ (Text, Text) -> PackageSet (Prod (PackageFetcher : VersionSource : r))-fromGitHub e p = fetchGitHub (sourceGitHub e p) p+fromGitHub e (owner, repo) = fromGitHub' e (owner, repo, id) +-- | A synonym of 'fetchGitHub'' and 'sourceGitHub'+fromGitHub' ::+ PackageSet (Prod r) ->+ (Text, Text, NixFetcher Fresh -> NixFetcher Fresh) ->+ PackageSet+ (Prod (PackageFetcher : VersionSource : r))+fromGitHub' e p@(owner, repo, _) = fetchGitHub' (sourceGitHub e (owner, repo)) p+ -- | A synonym of 'fetchPypi' and 'sourcePypi' fromPypi :: PackageSet (Prod r) ->@@ -239,7 +259,7 @@ -- | git url Text -> PackageSet (Prod (VersionSource : r))-sourceGit e vurl = src e Git {..}+sourceGit e _vurl = src e Git {..} -- | This package follows the latest pypi release sourcePypi ::@@ -247,7 +267,7 @@ -- | pypi name Text -> PackageSet (Prod (VersionSource : r))-sourcePypi e pypi = src e Pypi {..}+sourcePypi e _pypi = src e Pypi {..} -- | This package follows the version of an Arch Linux package sourceArchLinux ::@@ -255,7 +275,7 @@ -- | package name in Arch Linux repo Text -> PackageSet (Prod (VersionSource : r))-sourceArchLinux e archpkg = src e ArchLinux {..}+sourceArchLinux e _archpkg = src e ArchLinux {..} -- | This package follows the version of an Aur package sourceAur ::@@ -263,14 +283,14 @@ -- | package name in Aur Text -> PackageSet (Prod (VersionSource : r))-sourceAur e aur = src e Aur {..}+sourceAur e _aur = src e Aur {..} -- | This package follows a pinned version sourceManual :: PackageSet (Prod r) -> Text -> PackageSet (Prod (VersionSource : r))-sourceManual e manual = src e Manual {..}+sourceManual e _manual = src e Manual {..} -- | This package follows the version of a repology package sourceRepology ::@@ -288,8 +308,26 @@ -- | owner and repo (Text, Text) -> PackageSet (Prod (PackageFetcher : r))-fetchGitHub e p = fetch e $ gitHubFetcher p+fetchGitHub e (owner, repo) = fetchGitHub' e (owner, repo, id) +-- | This package is fetched from a github repo+--+-- Similar to 'fetchGitHub', but allows a modifier to the fetcher.+-- For example, you can enable fetch submodules like:+--+-- @+-- define $ package "qliveplayer" `sourceGitHub` ("IsoaSFlus", "QLivePlayer") `fetchGitHub'` ("IsoaSFlus", "QLivePlayer", fetchSubmodules .~ True)+-- @+fetchGitHub' ::+ PackageSet (Prod r) ->+ -- | owner and repo+ ( Text,+ Text,+ NixFetcher Fresh -> NixFetcher Fresh+ ) ->+ PackageSet (Prod (PackageFetcher : r))+fetchGitHub' e (owner, repo, f) = fetch e $ f . gitHubFetcher (owner, repo)+ -- | This package is fetched from a file in github release fetchGitHubRelease :: PackageSet (Prod r) ->@@ -312,7 +350,18 @@ -- | git url Text -> PackageSet (Prod (PackageFetcher : r))-fetchGit e = fetch e . gitFetcher+fetchGit e u = fetchGit' e (u, id)++-- | This package is fetched from git+--+-- Similar to 'fetchGit', but allows a modifier to the fetcher.+-- See 'fetchGitHub'' for a concret example.+fetchGit' ::+ PackageSet (Prod r) ->+ -- | git url+ (Text, NixFetcher Fresh -> NixFetcher Fresh) ->+ PackageSet (Prod (PackageFetcher : r))+fetchGit' e (u, f) = fetch e $ f . gitFetcher u -- | This package is fetched from url fetchUrl ::
src/NvFetcher/ShakeExtras.hs view
@@ -27,15 +27,15 @@ where import Control.Concurrent.Extra-import Data.HashMap.Strict (HashMap)-import qualified Data.HashMap.Strict as HMap+import Data.Map.Strict (Map)+import qualified Data.Map.Strict as Map import Development.Shake import NvFetcher.Types -- | Values we use during the build. It's stored in 'shakeExtra' data ShakeExtras = ShakeExtras { versionChanges :: Var [VersionChange],- targetPackages :: HashMap PackageKey Package+ targetPackages :: Map PackageKey Package } -- | Get our values from shake@@ -46,7 +46,7 @@ _ -> fail "ShakeExtras is missing!" -- | Create an empty 'ShakeExtras' from packages to build-initShakeExtras :: HashMap PackageKey Package -> IO ShakeExtras+initShakeExtras :: Map PackageKey Package -> IO ShakeExtras initShakeExtras targetPackages = do versionChanges <- newVar mempty pure ShakeExtras {..}@@ -55,17 +55,17 @@ getAllPackageKeys :: Action [PackageKey] getAllPackageKeys = do ShakeExtras {..} <- getShakeExtras- pure $ HMap.keys targetPackages+ pure $ Map.keys targetPackages -- | Find a package given its key lookupPackage :: PackageKey -> Action (Maybe Package) lookupPackage key = do ShakeExtras {..} <- getShakeExtras- pure $ HMap.lookup key targetPackages+ pure $ Map.lookup key targetPackages -- | Check if we need build this package isPackageKeyTarget :: PackageKey -> Action Bool-isPackageKeyTarget k = HMap.member k . targetPackages <$> getShakeExtras+isPackageKeyTarget k = Map.member k . targetPackages <$> getShakeExtras -- | Record version change of a package recordVersionChange :: PackageName -> Maybe Version -> Version -> Action ()
src/NvFetcher/Types.hs view
@@ -95,13 +95,13 @@ -- | The input of nvchecker data VersionSource- = GitHubRelease {owner :: Text, repo :: Text}- | Git {vurl :: Text}- | Pypi {pypi :: Text}- | ArchLinux {archpkg :: Text}- | Aur {aur :: Text}- | Manual {manual :: Text}- | Repology {repology :: Text, repo :: Text}+ = GitHubRelease {_owner :: Text, _repo :: Text}+ | Git {_vurl :: Text}+ | Pypi {_pypi :: Text}+ | ArchLinux {_archpkg :: Text}+ | Aur {_aur :: Text}+ | Manual {_manual :: Text}+ | Repology {_repology :: Text, _repo :: Text} deriving (Show, Typeable, Eq, Ord, Generic, Hashable, Binary, NFData) -- | The result of running nvchecker@@ -123,15 +123,15 @@ -- | If the package is prefetched, then we can obtain the SHA256 data NixFetcher (k :: Prefetch) = FetchGit- { furl :: Text,- rev :: Version,- branch :: Maybe Text,- deepClone :: Bool,- fetchSubmodules :: Bool,- leaveDotGit :: Bool,- sha256 :: PrefetchResult k+ { _furl :: Text,+ _rev :: Version,+ _branch :: Maybe Text,+ _deepClone :: Bool,+ _fetchSubmodules :: Bool,+ _leaveDotGit :: Bool,+ _sha256 :: PrefetchResult k }- | FetchUrl {furl :: Text, sha256 :: PrefetchResult k}+ | FetchUrl {_furl :: Text, _sha256 :: PrefetchResult k} deriving (Typeable, Generic) -- | Prefetch status@@ -174,9 +174,9 @@ -- /i.e. can only be concatenated with other strings,/ -- /in case we can't check the equality between fetcher functions./ data Package = Package- { pname :: PackageName,- pversion :: VersionSource,- pfetcher :: PackageFetcher+ { _pname :: PackageName,+ _pversion :: VersionSource,+ _pfetcher :: PackageFetcher } -- | Package key is the name of a package.
+ src/NvFetcher/Types/Lens.hs view
@@ -0,0 +1,18 @@+{-# LANGUAGE TemplateHaskell #-}++-- | Copyright: (c) 2021 berberman+-- SPDX-License-Identifier: MIT+-- Maintainer: berberman <berberman@yandex.com>+-- Stability: experimental+-- Portability: portable+-- Lenses for "NvFetcher.Types"+module NvFetcher.Types.Lens where++import Lens.Micro.TH+import NvFetcher.Types++makeLenses ''VersionSource++makeLenses ''NixFetcher++makeLenses ''Package