cachix 0.1.3 → 0.2.0
raw patch · 19 files changed
+537/−246 lines, 19 filesdep +netrcdep +servant-conduitdep +temporarydep −bifunctorsdep −resourcetdep −servant-streaming-clientdep ~cachix-apidep ~megaparsecdep ~servant
Dependencies added: netrc, servant-conduit, temporary
Dependencies removed: bifunctors, resourcet, servant-streaming-client, streaming
Dependency ranges changed: cachix-api, megaparsec, servant, servant-client, servant-client-core
Files
- CHANGELOG.md +82/−0
- ChangeLog.md +0/−61
- cachix.cabal +15/−11
- src/Cachix/Client.hs +2/−1
- src/Cachix/Client/Commands.hs +111/−101
- src/Cachix/Client/Config.hs +17/−3
- src/Cachix/Client/Exception.hs +20/−0
- src/Cachix/Client/InstallationMode.hs +110/−52
- src/Cachix/Client/NetRc.hs +60/−0
- src/Cachix/Client/NixConf.hs +20/−10
- src/Cachix/Client/OptionsParser.hs +27/−7
- test/NetRcSpec.hs +59/−0
- test/NixConfSpec.hs +1/−0
- test/data/add.input +1/−0
- test/data/add.output +3/−0
- test/data/empty.input +0/−0
- test/data/empty.output +3/−0
- test/data/override.input +3/−0
- test/data/override.output +3/−0
+ CHANGELOG.md view
@@ -0,0 +1,82 @@+# Changelog++All notable changes to this project will be documented in this file.++The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)+and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).++## Unreleased++## [0.2.0] - 2019-03-04++### Added++- #71 new command `cachix generate-keypair` with+ clearer instructions how to handle the secrets @domenkozar+- #24 #168 Private binary cache support @domenkozar @roberth++### Changed+- #158 #160 #166 #91 Improve NixOS experience by+ writing out nixos modules @domenkozar+- #170 Get rid of amazonka dependency @domenkozar++## [0.1.3] - 2018-11-26++### Changed++- #77 retry transient HTTP exceptions @domenkozar+- #151 prevent mingled store paths output @domenkozar+- #141 prevent unncessary warning about /etc/nix missing @domenkozar+- #142 ditch hpack @domenkozar++## [0.1.2] - 2018-09-27++### Changed++- #132 error handling for readProcess invocations @domenkozar+- #130 only warn about not supporting groups if user is not trusted @domenkozar+- #128 Generate https://cache.nixos.org when run as root on NixOS @yegortimoschenko+- #121 bail out if narSize is 0 @domenkozar+- #123 support passing --config @domenkozar+- #123 no more spurious warning messages when using "cachix use" @domenkozar+- #105 pass https://cache.nixos.org explicitly @domenkozar++## [0.1.1] - 2018-08-03++### Added++- #102 Nix 1.0 support @domenkozar++### Changed++- #105 Always specify cachix.nixos.org on NixOS @domenkozar+- #98, #95 Use LTS-12.X and Servant 0.14.1 @domenkozar++## [0.1.0.2] - 2018-07-06++### Changed++- #95 Upgrade to servant-0.14.1 @domenkozar++## [0.1.0.1] - 2018-07-05++### Changed++- #92 Add build-tool-depends where needed @domenkozar+- #90 HLint fixes @domenkozar+- #86 Improve Cabal description and synopsis @domenkozar+- #87 Support fsnotify 0.3.x @domenkozar++## [0.1.0.0] - 2018-07-01++### Added++- Initial release @domenkozar++[Unreleased]: https://github.com/cachix/cachix/compare/v0.1.4...HEAD+[0.2.0]: https://github.com/cachix/cachix/compare/v0.1.3...v0.2.0+[0.1.3]: https://github.com/cachix/cachix/compare/v0.1.2...v0.1.3+[0.1.2]: https://github.com/cachix/cachix/compare/v0.1.1...v0.1.2+[0.1.1]: https://github.com/cachix/cachix/compare/v0.1.0.2...v0.1.1+[0.1.0.2]: https://github.com/cachix/cachix/compare/v0.1.0.1...v0.1.0.2+[0.1.0.1]: https://github.com/cachix/cachix/compare/v0.1.0.0...v0.1.0.1
− ChangeLog.md
@@ -1,61 +0,0 @@-# Changelog--All notable changes to this project will be documented in this file.--The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)-and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).--## Unreleased--## [0.1.3] - 2018-11-26--### Changed--- #77 retry transient HTTP exceptions @domenkozar-- #151 prevent mingled store paths output @domenkozar-- #141 prevent unncessary warning about /etc/nix missing @domenkozar-- #142 ditch hpack @domenkozar--## [0.1.2] - 2018-09-27--### Changed--- #132 error handling for readProcess invocations @domenkozar-- #130 only warn about not supporting groups if user is not trusted @domenkozar-- #128 Generate https://cache.nixos.org when run as root on NixOS @yegortimoschenko-- #121 bail out if narSize is 0 @domenkozar-- #123 support passing --config @domenkozar-- #123 no more spurious warning messages when using "cachix use" @domenkozar-- #105 pass https://cache.nixos.org explicitly @domenkozar--## [0.1.1] - 2018-08-03--### Added--- #102 Nix 1.0 support @domenkozar--### Changed--- #105 Always specify cachix.nixos.org on NixOS @domenkozar-- #98, #95 Use LTS-12.X and Servant 0.14.1 @domenkozar--## [0.1.0.2] - 2018-07-06--### Changed--- #95 Upgrade to servant-0.14.1 @domenkozar--## [0.1.0.1] - 2018-07-05--### Changed--- #92 Add build-tool-depends where needed @domenkozar-- #90 HLint fixes @domenkozar-- #86 Improve Cabal description and synopsis @domenkozar-- #87 Support fsnotify 0.3.x @domenkozar--## [0.1.0.0] - 2018-07-01--### Added--- Initial release @domenkozar
cachix.cabal view
@@ -1,6 +1,6 @@ cabal-version: 2.2 name: cachix-version: 0.1.3+version: 0.2.0 synopsis: Command line client for Nix binary cache hosting https://cachix.org homepage: https://github.com/cachix/cachix#readme bug-reports: https://github.com/cachix/cachix/issues@@ -10,9 +10,10 @@ license: Apache-2.0 license-file: LICENSE build-type: Simple-+data-files: *.input, *.output+data-dir: test/data extra-source-files:- ChangeLog.md+ CHANGELOG.md README.md common defaults@@ -31,7 +32,9 @@ Cachix.Client.Commands Cachix.Client.Config Cachix.Client.Env+ Cachix.Client.Exception Cachix.Client.InstallationMode+ Cachix.Client.NetRc Cachix.Client.NixConf Cachix.Client.NixVersion Cachix.Client.OptionsParser@@ -48,7 +51,6 @@ , base >=4.7 && <5 , base16-bytestring , base64-bytestring- , bifunctors , bytestring , cachix-api , conduit >=1.3.0@@ -67,22 +69,21 @@ , http-conduit , http-types , lzma-conduit- , megaparsec+ , megaparsec >= 7.0.0 , memory , mmorph+ , netrc , optparse-applicative , process , protolude- , resourcet , retry , safe-exceptions- , servant >=0.14.1+ , servant >=0.15 , servant-auth , servant-auth-client >=0.3.3.0- , servant-client >=0.14- , servant-client-core >=0.14.1- , servant-streaming-client >=0.3.0.0- , streaming+ , servant-client >=0.15+ , servant-client-core >=0.15+ , servant-conduit , text , unix , uri-bytestring@@ -110,6 +111,7 @@ main-is: Main.hs other-modules: InstallationModeSpec+ NetRcSpec NixConfSpec NixVersionSpec Spec@@ -121,6 +123,8 @@ base >=4.7 && <5 , cachix , cachix-api+ , directory , hspec , protolude , here+ , temporary
src/Cachix/Client.hs view
@@ -16,6 +16,7 @@ case command of AuthToken token -> Commands.authtoken env token Create name -> Commands.create env name+ GenerateKeypair name -> Commands.generateKeypair env name Push name paths watchStore -> Commands.push env name paths watchStore- Use name shouldEchoNixOS -> Commands.use env name shouldEchoNixOS+ Use name useOptions -> Commands.use env name useOptions Version -> putText cachixVersion
src/Cachix/Client/Commands.hs view
@@ -5,6 +5,7 @@ module Cachix.Client.Commands ( authtoken , create+ , generateKeypair , push , use ) where@@ -13,40 +14,40 @@ import Control.Concurrent (threadDelay) import qualified Control.Concurrent.QSem as QSem import Control.Concurrent.Async (mapConcurrently)-import Control.Monad (forever)+import Control.Monad (forever, (>=>)) import Control.Exception.Safe (MonadThrow, throwM)-import Control.Monad.Trans.Resource (ResourceT) import Control.Retry (recoverAll, RetryStatus(rsIterNumber)) import qualified Data.ByteString.Base64 as B64 import Data.Conduit import Data.Default (def) import Data.Conduit.Process-import qualified Data.Conduit.List as CL import Data.Conduit.Lzma ( compress ) import Data.IORef import Data.List ( isSuffixOf ) import Data.Maybe ( fromJust ) import Data.String.Here import qualified Data.Text as T-import Network.HTTP.Types (status404)+import Network.HTTP.Types (status404, status401) import Protolude import Servant.API-import Servant.Client import Servant.Auth () import Servant.Auth.Client-import Servant.Streaming.Client () import Servant.API.Generic import Servant.Client.Generic+import Servant.Client.Streaming+import Servant.Conduit () import System.Directory ( doesFileExist ) import System.FSNotify import System.IO ( stdin, hIsTerminalDevice ) import System.Process ( readProcess ) import System.Environment ( lookupEnv )-import qualified Streaming.Prelude as S import qualified Cachix.Api as Api+import Cachix.Api.Error import Cachix.Api.Signing (fingerprint, passthroughSizeSink, passthroughHashSink) import qualified Cachix.Types.NarInfoCreate as Api+import qualified Cachix.Types.BinaryCacheCreate as Api+import qualified Cachix.Types.SigningKeyCreate as SigningKeyCreate import Cachix.Client.Config ( Config(..) , BinaryCacheConfig(..) , writeConfig@@ -54,7 +55,8 @@ ) import qualified Cachix.Client.Config as Config import Cachix.Client.Env ( Env(..) )-import Cachix.Client.OptionsParser ( CachixOptions(..) )+import Cachix.Client.Exception ( CachixException(..) )+import Cachix.Client.OptionsParser ( CachixOptions(..), UseOptions(..) ) import Cachix.Client.InstallationMode import Cachix.Client.NixVersion ( getNixVersion ) import qualified Cachix.Client.NixConf as NixConf@@ -67,40 +69,42 @@ cachixBCClient :: Text -> Api.BinaryCacheAPI (AsClientT ClientM) cachixBCClient name = fromServant $ Api.cache cachixClient name +cachixBCStreamingClient :: Text -> Api.BinaryCacheStreamingAPI (AsClientT ClientM)+cachixBCStreamingClient name = fromServant $ client (Proxy :: Proxy Api.BinaryCachStreamingServantAPI) name+ authtoken :: Env -> Text -> IO () authtoken env token = do -- TODO: check that token actually authenticates! writeConfig (configPath (cachixoptions env)) $ case config env of- Just config -> config { authToken = token }+ Just config -> config { authToken = Token (toS token) } Nothing -> mkConfig token putStrLn ([hereLit|-Continue by creating a binary cache with:-- $ cachix create <name>--and share it with others over https://<name>.cachix.org+Continue by creating a binary cache at https://cachix.org |] :: Text) create :: Env -> Text -> IO ()-create Env { config = Nothing } _ = throwIO $ NoConfig "start with: $ cachix authtoken <token>"-create env@Env { config = Just config } name = do+create _ _ =+ throwIO $ DeprecatedCommand "Create command has been deprecated. Please visit https://cachix.org to create a binary cache."++generateKeypair :: Env -> Text -> IO ()+generateKeypair Env { config = Nothing } _ = throwIO $ NoConfig "Start with visiting https://cachix.org and copying the token to $ cachix authtoken <token>"+generateKeypair env@Env { config = Just config } name = do (PublicKey pk, SecretKey sk) <- createKeypair - let bc = Api.BinaryCacheCreate $ toS $ B64.encode pk- res <- (`runClientM` clientenv env) $ Api.create (cachixBCClient name) (Token (toS (authToken config))) bc- case res of- -- TODO: handle all kinds of errors- Left err -> panic $ show err- Right _ -> do- -- write signing key to config- let signingKey = toS $ B64.encode sk- bcc = BinaryCacheConfig name signingKey+ let signingKey = toS $ B64.encode sk+ signingKeyCreate = SigningKeyCreate.SigningKeyCreate (toS $ B64.encode pk)+ bcc = BinaryCacheConfig name signingKey - writeConfig (configPath (cachixoptions env)) $- config { binaryCaches = binaryCaches config <> [bcc] }+ -- we first validate if key can be added to the binary cache+ escalate =<< ((`runClientM` clientenv env) $ Api.createKey (cachixBCClient name) (authToken config) signingKeyCreate) - putStrLn ([iTrim|-Signing key has been saved on your local machine. To populate+ -- if key was successfully added, write it to the config+ -- TODO: this breaks if more than one key is added, see #27+ writeConfig (configPath (cachixoptions env)) $+ config { binaryCaches = binaryCaches config <> [bcc] }++ putStrLn ([iTrim|+Secret signing key has been saved in the file above. To populate your binary cache: $ nix-build | cachix push ${name}@@ -114,34 +118,46 @@ $ cachix use ${name} - |] :: Text)+IMPORTANT: Make sure to make a backup for the signing key above, as you have the only copy.+ |] :: Text) -use :: Env -> Text -> Bool -> IO ()-use env name shouldEchoNixOS = do+envToToken :: Env -> Token+envToToken env =+ maybe (Token "") authToken (config env)++notAuthenticatedBinaryCache :: CachixException+notAuthenticatedBinaryCache =+ AccessDeniedBinaryCache "You must first authenticate using: $ cachix authtoken <token>"++accessDeniedBinaryCache :: Text -> CachixException+accessDeniedBinaryCache name =+ AccessDeniedBinaryCache $ "You don't seem to have API access to binary cache " <> name++use :: Env -> Text -> UseOptions -> IO ()+use env name useOptions = do -- 1. get cache public key- res <- (`runClientM` clientenv env) $ Api.get (cachixBCClient name)+ res <- (`runClientM` clientenv env) $ Api.get (cachixBCClient name) (envToToken env) case res of- -- TODO: handle 404- Left err -> panic $ show err+ Left err | isErr err status401 && isJust (config env) -> throwM $ accessDeniedBinaryCache name+ | isErr err status401 -> throwM notAuthenticatedBinaryCache+ | isErr err status404 -> throwM $ BinaryCacheNotFound $ "Binary cache" <> name <> " does not exist."+ | otherwise -> throwM err Right binaryCache -> do- eitherNixVersion <- getNixVersion- case eitherNixVersion of- Left err -> panic err- Right nixVersion -> do- user <- getUser- nc <- NixConf.read NixConf.Global- isTrusted <- isTrustedUser $ NixConf.readLines (catMaybes [nc]) NixConf.isTrustedUsers- isNixOS <- doesFileExist "/etc/NIXOS"- let nixEnv = NixEnv- { nixVersion = nixVersion- , isRoot = user == "root"- , isTrusted = isTrusted- , isNixOS = isNixOS- }- addBinaryCache binaryCache $- if shouldEchoNixOS- then EchoNixOS nixVersion- else getInstallationMode nixEnv+ nixVersion <- escalateAs UnsupportedNixVersion =<< getNixVersion+ user <- getUser+ nc <- NixConf.read NixConf.Global+ isTrusted <- isTrustedUser $ NixConf.readLines (catMaybes [nc]) NixConf.isTrustedUsers+ isNixOS <- doesFileExist "/etc/NIXOS"+ let nixEnv = NixEnv+ { nixVersion = nixVersion+ , isRoot = user == "root"+ , isTrusted = isTrusted+ , isNixOS = isNixOS+ }+ addBinaryCache (config env) binaryCache useOptions $+ if useNixOS useOptions+ then EchoNixOS nixVersion+ else getInstallationMode nixEnv -- TODO: lots of room for perfomance improvements@@ -206,11 +222,14 @@ -- TODO: query also cache.nixos.org? server-side? res <- (`runClientM` clientenv env) $ Api.narinfoHead (cachixBCClient name)+ (envToToken env) (Api.NarInfoC storeHash) case res of Right NoContent -> pass -- we're done as store path is already in the cache Left err | isErr err status404 -> go sk retrystatus- | otherwise -> escalate $ void res+ | isErr err status401 && isJust (config env) -> throwM $ accessDeniedBinaryCache name+ | isErr err status401 -> throwM notAuthenticatedBinaryCache+ | otherwise -> throwM err where retryText :: RetryStatus -> Text retryText retrystatus =@@ -238,58 +257,55 @@ .| passthroughSizeSink fileSizeRef .| passthroughHashSink fileHashRef - conduitToStreaming :: S.Stream (S.Of ByteString) (ResourceT IO) ()- conduitToStreaming = runConduit (transPipe lift stream' .| CL.mapM_ S.yield) -- for now we need to use letsencrypt domain instead of cloudflare due to its upload limits let newClientEnv = (clientenv env) { baseUrl = (baseUrl (clientenv env)) { baseUrlHost = toS name <> "." <> baseUrlHost (baseUrl (clientenv env))} }- NoContent <- escalate <=< (`runClientM` newClientEnv) $ Api.createNar- (cachixBCClient name)- (contentType (Proxy :: Proxy Api.XNixNar), conduitToStreaming)- closeStdout- exitcode <- waitForStreamingProcess cph- -- TODO: print process stderr?- when (exitcode /= ExitSuccess) $ throwM $ NarStreamingError exitcode $ show cmd+ void $ (`withClientM` newClientEnv)+ (Api.createNar (cachixBCStreamingClient name) stream')+ $ escalate >=> \NoContent -> do+ closeStdout+ exitcode <- waitForStreamingProcess cph+ -- TODO: print process stderr?+ when (exitcode /= ExitSuccess) $ throwM $ NarStreamingError exitcode $ show cmd - narSize <- readIORef narSizeRef- narHashB16 <- readIORef narHashRef- fileHash <- readIORef fileHashRef- fileSize <- readIORef fileSizeRef+ narSize <- readIORef narSizeRef+ narHashB16 <- readIORef narHashRef+ fileHash <- readIORef fileHashRef+ fileSize <- readIORef fileSizeRef - -- TODO: #3: implement using pure haskell- narHash <- ("sha256:" <>) . T.strip . toS <$> readProcess "nix-hash" ["--type", "sha256", "--to-base32", toS narHashB16] mempty+ -- TODO: #3: implement using pure haskell+ narHash <- ("sha256:" <>) . T.strip . toS <$> readProcess "nix-hash" ["--type", "sha256", "--to-base32", toS narHashB16] mempty - deriverRaw <- T.strip . toS <$> readProcess "nix-store" ["-q", "--deriver", toS storePath] mempty- let deriver = if deriverRaw == "unknown-deriver"- then deriverRaw- else T.drop 11 deriverRaw+ deriverRaw <- T.strip . toS <$> readProcess "nix-store" ["-q", "--deriver", toS storePath] mempty+ let deriver = if deriverRaw == "unknown-deriver"+ then deriverRaw+ else T.drop 11 deriverRaw - references <- sort . T.lines . T.strip . toS <$> readProcess "nix-store" ["-q", "--references", toS storePath] mempty+ references <- sort . T.lines . T.strip . toS <$> readProcess "nix-store" ["-q", "--references", toS storePath] mempty - let- fp = fingerprint storePath narHash narSize references- sig = dsign sk fp- nic = Api.NarInfoCreate- { cStoreHash = storeHash- , cStoreSuffix = storeSuffix- , cNarHash = narHash- , cNarSize = narSize- , cFileSize = fileSize- , cFileHash = fileHash- , cReferences = fmap (T.drop 11) references- , cDeriver = deriver- , cSig = toS $ B64.encode $ unSignature sig- }+ let+ fp = fingerprint storePath narHash narSize references+ sig = dsign sk fp+ nic = Api.NarInfoCreate+ { cStoreHash = storeHash+ , cStoreSuffix = storeSuffix+ , cNarHash = narHash+ , cNarSize = narSize+ , cFileSize = fileSize+ , cFileHash = fileHash+ , cReferences = fmap (T.drop 11) references+ , cDeriver = deriver+ , cSig = toS $ B64.encode $ unSignature sig+ } - escalate $ Api.isNarInfoCreateValid nic+ escalate $ Api.isNarInfoCreateValid nic - -- Upload narinfo with signature- NoContent <- escalate <=< (`runClientM` clientenv env) $ Api.createNarinfo- (cachixBCClient name)- (Api.NarInfoC storeHash)- nic- return ()+ -- Upload narinfo with signature+ escalate <=< (`runClientM` clientenv env) $ Api.createNarinfo+ (cachixBCClient name)+ (Api.NarInfoC storeHash)+ nic -- Utils@@ -298,12 +314,6 @@ -- Catches all exceptions except skipAsyncExceptions retryPath :: (RetryStatus -> IO a) -> IO a retryPath = recoverAll def--escalate :: (Exception exc, MonadThrow m) => Either exc a -> m a-escalate = escalateAs identity--escalateAs :: (Exception exc, MonadThrow m) => (l -> exc) -> Either l a -> m a-escalateAs f = either (throwM . f) pure mapConcurrentlyBounded :: Traversable t => Int -> (a -> IO b) -> t a -> IO (t b) mapConcurrentlyBounded bound action items = do
src/Cachix/Client/Config.hs view
@@ -11,7 +11,9 @@ import Dhall hiding ( Text ) import Dhall.Pretty ( prettyExpr )+import Dhall.Core (Expr(..), Chunks(..)) import GHC.Generics ( Generic )+import Servant.Auth.Client import System.Directory ( doesFileExist, createDirectoryIfMissing , getXdgDirectory, XdgDirectory(..) )@@ -27,13 +29,25 @@ } deriving (Show, Generic, Interpret, Inject) data Config = Config- { authToken :: Text+ { authToken :: Token , binaryCaches :: [BinaryCacheConfig] } deriving (Show, Generic, Interpret, Inject) +instance Interpret Token where+ autoWith _ = Type+ { extract = \(TextLit (Chunks [] t)) -> pure (Token (toS t))+ , expected = Text+ }++instance Inject Token where+ injectWith _ = InputType+ { embed = TextLit . Chunks [] . toS . getToken+ , declared = Text+ }+ mkConfig :: Text -> Config-mkConfig authtoken = Config- { authToken = authtoken+mkConfig token = Config+ { authToken = Token (toS token) , binaryCaches = [] }
+ src/Cachix/Client/Exception.hs view
@@ -0,0 +1,20 @@+module Cachix.Client.Exception (CachixException(..)) where++import Protolude++data CachixException+ = UnsupportedNixVersion Text+ | UserEnvNotSet Text+ | MustBeRoot Text+ | NixOSInstructions Text+ | AmbiguousInput Text+ | NoInput Text+ | NoConfig Text+ | NetRcParseError Text+ | NarStreamingError ExitCode Text+ | DeprecatedCommand Text+ | AccessDeniedBinaryCache Text+ | BinaryCacheNotFound Text+ deriving (Show, Typeable)++instance Exception CachixException
src/Cachix/Client/InstallationMode.hs view
@@ -2,7 +2,6 @@ module Cachix.Client.InstallationMode ( InstallationMode(..) , NixEnv(..)- , CachixException(..) , getInstallationMode , addBinaryCache , isTrustedUser@@ -12,24 +11,16 @@ import Protolude import Data.String.Here import qualified Data.Text as T+import Cachix.Client.Config (Config)+import Cachix.Client.Exception (CachixException(..))+import qualified Cachix.Client.NetRc as NetRc import qualified Cachix.Client.NixConf as NixConf import Cachix.Client.NixVersion ( NixVersion(..) )+import Cachix.Client.OptionsParser ( UseOptions(..) ) import Cachix.Api as Api-import System.Directory ( getPermissions, writable )+import System.Directory ( getPermissions, writable, createDirectoryIfMissing ) import System.Environment ( lookupEnv )--data CachixException- = UnsupportedNixVersion Text- | UserEnvNotSet Text- | MustBeRoot Text- | NixOSInstructions Text- | AmbiguousInput Text- | NoInput Text- | NoConfig Text- | NarStreamingError ExitCode Text- deriving (Show, Typeable)--instance Exception CachixException+import System.FilePath ( replaceFileName, (</>) ) data NixEnv = NixEnv { nixVersion :: NixVersion@@ -57,49 +48,116 @@ -- | Add a Binary cache to nix.conf, print nixos config or fail-addBinaryCache :: Api.BinaryCache -> InstallationMode -> IO ()-addBinaryCache Api.BinaryCache{..} (EchoNixOS _) = do- putText [iTrim|-nix = {- binaryCaches = [- "https://cache.nixos.org/"- "${uri}"- ];- binaryCachePublicKeys = [- ${T.intercalate " " (map (\s -> "\"" <> s <> "\"") publicSigningKeys)}- ];-};- |]- throwIO $ NixOSInstructions "Add above lines to your NixOS configuration file"-addBinaryCache Api.BinaryCache{..} (EchoNixOSWithTrustedUser _) = do- -- TODO: DRY- user <- getUser- putText [iTrim|-nix = {- binaryCaches = [- "https://cache.nixos.org/"- "${uri}"- ];- binaryCachePublicKeys = [- ${T.intercalate " " (map (\s -> "\"" <> s <> "\"") publicSigningKeys)}- ];- trustedUsers = [ "root" "${user}" ];-};- |]- throwIO $ NixOSInstructions "Add above lines to your NixOS configuration file"-addBinaryCache _ UntrustedRequiresSudo = throwIO $+addBinaryCache :: Maybe Config -> Api.BinaryCache -> UseOptions -> InstallationMode -> IO ()+addBinaryCache _ _ _ UntrustedRequiresSudo = throwIO $ MustBeRoot "Run command as root OR execute: $ echo \"trusted-users = root $USER\" | sudo tee -a /etc/nix/nix.conf && sudo pkill nix-daemon"-addBinaryCache _ Nix20RequiresSudo = throwIO $+addBinaryCache _ _ _ Nix20RequiresSudo = throwIO $ MustBeRoot "Run command as root OR upgrade to latest Nix - to be able to use it without root (recommended)"-addBinaryCache bc@Api.BinaryCache{..} (Install nixversion ncl) = do+addBinaryCache maybeConfig bc useOptions (EchoNixOS _) =+ nixosBinaryCache maybeConfig Nothing bc useOptions+addBinaryCache maybeConfig bc useOptions (EchoNixOSWithTrustedUser _) = do+ user <- getUser+ nixosBinaryCache maybeConfig (Just user) bc useOptions+addBinaryCache maybeConfig bc@Api.BinaryCache{..} _ (Install nixversion ncl) = do -- TODO: might need locking one day gnc <- NixConf.read NixConf.Global- lnc <- NixConf.read NixConf.Local- let final = if ncl == NixConf.Global then gnc else lnc- input = if ncl == NixConf.Global then [gnc] else [gnc, lnc]- NixConf.write nixversion ncl $ NixConf.add bc (catMaybes input) (fromMaybe (NixConf.NixConf []) final)+ (input, output) <- case ncl of+ NixConf.Global -> return ([gnc], gnc)+ NixConf.Local -> do+ lnc <- NixConf.read NixConf.Local+ return ([gnc, lnc], lnc)+ let nixconf = fromMaybe (NixConf.NixConf []) output+ netrcLocMaybe <- forM (guard $ not isPublic) $ const $ addPrivateBinaryCacheNetRC maybeConfig bc ncl+ let+ addNetRCLine :: NixConf.NixConf -> NixConf.NixConf+ addNetRCLine = fromMaybe identity $ do+ netrcLoc <- netrcLocMaybe :: Maybe FilePath+ -- We only add the netrc line for local user configs for now.+ -- On NixOS we assume it will be picked up from the default location.+ guard (ncl == NixConf.Local)+ pure (NixConf.setNetRC $ toS netrcLoc)+ NixConf.write nixversion ncl $ addNetRCLine $ NixConf.add bc (catMaybes input) nixconf filename <- NixConf.getFilename ncl putStrLn $ "Configured " <> uri <> " binary cache in " <> toS filename++nixosBinaryCache :: Maybe Config -> Maybe Text -> Api.BinaryCache -> UseOptions -> IO ()+nixosBinaryCache maybeConfig maybeUser bc@Api.BinaryCache{..} UseOptions { useNixOSFolder=baseDirectory } = do+ createDirectoryIfMissing True $ toS toplevel+ writeFile (toS glueModuleFile) glueModule+ writeFile (toS cacheModuleFile) cacheModule+ unless isPublic $ void $ addPrivateBinaryCacheNetRC maybeConfig bc NixConf.Global+ putText instructions+ where+ configurationNix :: Text+ configurationNix = toS $ toS baseDirectory </> "configuration.nix"+ namespace :: Text+ namespace = "cachix"+ toplevel :: Text+ toplevel = toS $ toS baseDirectory </> toS namespace+ glueModuleFile :: Text+ glueModuleFile = toplevel <> ".nix"+ cacheModuleFile :: Text+ cacheModuleFile = toplevel <> "/" <> toS name <> ".nix"++ extra :: Text+ extra = maybe "" (\user -> [i|trustedUsers = [ "root" "${user}" ];|]) maybeUser++ instructions :: Text+ instructions = [iTrim|+Cachix configuration written to ${glueModuleFile}.+Binary cache ${name} configuration written to ${cacheModuleFile}.++To start using cachix add the following to your ${configurationNix}:++ imports = [ ./cachix.nix ];++Then run:++ $ nixos-rebuild switch+ |]++ glueModule :: Text+ glueModule = [i|+# WARN: this file will get overwritten by $ cachix use <name>+{ pkgs, lib, ... }:++let+ folder = ./cachix;+ toImport = name: value: folder + ("/" + name);+ filterCaches = key: value: value == "regular" && lib.hasSuffix ".nix" key;+ imports = lib.mapAttrsToList toImport (lib.filterAttrs filterCaches (builtins.readDir folder));+in {+ inherit imports;+ nix.binaryCaches = ["https://cache.nixos.org/"];+}+ |]++ cacheModule :: Text+ cacheModule = [i|+{+ nix = {+ binaryCaches = [+ "${uri}"+ ];+ binaryCachePublicKeys = [+ ${T.intercalate " " (map (\s -> "\"" <> s <> "\"") publicSigningKeys)}+ ];+ ${extra}+ };+}+ |]++-- TODO: allow overriding netrc location+addPrivateBinaryCacheNetRC :: Maybe Config -> Api.BinaryCache -> NixConf.NixConfLoc -> IO FilePath+addPrivateBinaryCacheNetRC maybeConfig bc nixconf = do+ filename <- (`replaceFileName` "netrc") <$> NixConf.getFilename nixconf+ case maybeConfig of+ Nothing -> panic "Run $ cachix authtoken <token>"+ Just config -> do+ let netrcfile = fromMaybe filename Nothing -- TODO: get netrc from nixconf+ NetRc.add config [bc] netrcfile+ putErrText $ "Configured private read access credentials in " <> toS filename+ pure filename isTrustedUser :: [Text] -> IO Bool isTrustedUser users = do
+ src/Cachix/Client/NetRc.hs view
@@ -0,0 +1,60 @@+-- Deals with adding private caches to netrc+module Cachix.Client.NetRc+ ( add+ ) where++import qualified Data.ByteString as BS+import Data.List (nubBy)+import qualified Data.Text as T+import Network.NetRc+import Protolude+import System.Directory ( doesFileExist )+import Servant.Auth.Client (getToken)++import qualified Cachix.Api as Api+import Cachix.Api.Error (escalateAs)+import Cachix.Client.Config (Config, authToken)+import Cachix.Client.Exception (CachixException(NetRcParseError))+++-- | Add a list of binary caches to netrc under `filename`.+-- Makes sure there are no duplicate entries (using domain as a key).+-- If file under filename doesn't exist it's created.+add :: Config+ -> [Api.BinaryCache]+ -> FilePath+ -> IO ()+add config binarycaches filename = do+ doesExist <- doesFileExist filename+ netrc <- if doesExist+ then BS.readFile filename >>= parse+ else return $ NetRc [] []+ BS.writeFile filename $ netRcToByteString $ uniqueAppend netrc+ where+ parse :: ByteString -> IO NetRc+ parse contents = escalateAs (NetRcParseError . show) $ parseNetRc filename contents++ -- O(n^2) but who cares?+ uniqueAppend :: NetRc -> NetRc+ uniqueAppend (NetRc hosts macdefs) =+ let+ f :: NetRcHost -> NetRcHost -> Bool+ f x y = nrhName x == nrhName y+ in NetRc (nubBy f (new ++ hosts)) macdefs++ new :: [NetRcHost]+ new = map mkHost $ filter (not . Api.isPublic) binarycaches++ mkHost :: Api.BinaryCache -> NetRcHost+ mkHost bc = NetRcHost+ { nrhName = toS $ stripPrefix "http://" $ stripPrefix "https://" (Api.uri bc)+ , nrhLogin = ""+ , nrhPassword = getToken (authToken config)+ , nrhAccount = ""+ , nrhMacros = []+ }+ where+ -- stripPrefix that either strips or returns the same string+ stripPrefix :: Text -> Text -> Text+ stripPrefix prefix str =+ maybe str identity $ T.stripPrefix prefix str
src/Cachix/Client/NixConf.hs view
@@ -29,10 +29,11 @@ , defaultSigningKey , substitutersKey , trustedPublicKeysKey+ , setNetRC ) where import Data.Char (isSpace)-import Data.Text (unwords, unlines)+import qualified Data.Text as T import Data.List (nub) import Protolude import qualified Text.Megaparsec as Mega@@ -49,6 +50,7 @@ = Substituters [Text] | TrustedUsers [Text] | TrustedPublicKeys [Text]+ | NetRcFile Text | Other Text deriving (Show, Eq) @@ -98,12 +100,13 @@ defaultSigningKey = "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" render :: NixVersion -> NixConf -> Text-render nixversion (NixConf nixconflines) = unlines $ fmap go nixconflines+render nixversion (NixConf nixconflines) = T.unlines $ fmap go nixconflines where go :: NixConfLine -> Text- go (Substituters xs) = substitutersKey nixversion <> " = " <> unwords xs- go (TrustedUsers xs) = "trusted-users = " <> unwords xs- go (TrustedPublicKeys xs) = trustedPublicKeysKey nixversion <> " = " <> unwords xs+ go (Substituters xs) = substitutersKey nixversion <> " = " <> T.unwords xs+ go (TrustedUsers xs) = "trusted-users = " <> T.unwords xs+ go (TrustedPublicKeys xs) = trustedPublicKeysKey nixversion <> " = " <> T.unwords xs+ go (NetRcFile filename) = "netrc-file = " <> filename go (Other line) = line substitutersKey :: NixVersion -> Text@@ -132,7 +135,7 @@ result <- parse <$> readFile filename case result of Left err -> do- putStrLn (Mega.parseErrorPretty err)+ putStrLn (Mega.errorBundlePretty err) panic $ toS filename <> " failed to parse, please copy the above error and contents of nix.conf and open an issue at https://github.com/cachix/cachix" Right conf -> return $ Just conf @@ -141,6 +144,11 @@ nc <- f <$> read ncl write nixversion ncl nc +setNetRC :: Text -> NixConf -> NixConf+setNetRC netrc (NixConf nc) = NixConf $ filter noNetRc nc ++ [NetRcFile netrc]+ where noNetRc (NetRcFile _) = False+ noNetRc _ = True+ data NixConfLoc = Global | Local deriving (Show, Eq) @@ -163,25 +171,27 @@ _ <- many (char ' ') _ <- char '=' _ <- many (char ' ')- values <- Mega.sepBy1 (many (satisfy (not . isSpace))) (some (char ' '))+ values <- Mega.sepBy1 (many (Mega.satisfy (not . isSpace))) (some (char ' ')) _ <- many spaceChar return $ constr (fmap toS values) parseOther :: Parser NixConfLine-parseOther = Mega.try $ Other . toS <$> Mega.someTill anyChar (void eol <|> Mega.eof)+parseOther = Mega.try $ Other . toS <$> Mega.someTill Mega.anySingle (void eol <|> Mega.eof) parseAltLine :: Parser NixConfLine parseAltLine =- (const (Other "") <$> eol)+ (Other "" <$ eol) <|> parseLine Substituters "substituters" <|> parseLine TrustedPublicKeys "trusted-public-keys" <|> parseLine TrustedUsers "trusted-users" <|> parseLine TrustedPublicKeys "binary-cache-public-keys" <|> parseLine Substituters "binary-caches"+ -- NB: assume that space in this option means space in filename+ <|> parseLine (NetRcFile . T.concat) "netrc-file" <|> parseOther parser :: Parser NixConf parser = NixConf <$> many parseAltLine -parse :: Text -> Either (Mega.ParseError (Mega.Token Text) Void) NixConf+parse :: Text -> Either (Mega.ParseErrorBundle Text Void) NixConf parse = Mega.parse parser "nix.conf"
src/Cachix/Client/OptionsParser.hs view
@@ -2,6 +2,7 @@ module Cachix.Client.OptionsParser ( CachixCommand(..) , CachixOptions(..)+ , UseOptions(..) , BinaryCacheName , getOpts ) where@@ -51,26 +52,45 @@ data CachixCommand = AuthToken Text | Create BinaryCacheName+ | GenerateKeypair BinaryCacheName | Push BinaryCacheName [Text] Bool -- TODO: refactor to a record- | Use BinaryCacheName Bool --- TODO: refactor to a record+ | Use BinaryCacheName UseOptions | Version deriving Show +data UseOptions = UseOptions+ { useNixOS :: Bool+ , useNixOSFolder :: FilePath+ } deriving Show parserCachixCommand :: Parser CachixCommand parserCachixCommand = subparser $ command "authtoken" (infoH authtoken (progDesc "Configure token for authentication to cachix.org")) <>- command "create" (infoH create (progDesc "Create a new binary cache")) <>+ command "create" (infoH create (progDesc "DEPRECATED: Go to https://cachix.org instead")) <>+ command "generate-keypair" (infoH generateKeypair (progDesc "Generate keypair for an binary cache")) <> command "push" (infoH push (progDesc "Upload Nix store paths to the binary cache")) <> command "use" (infoH use (progDesc "Configure nix.conf to enable binary cache during builds")) where+ nameArg = strArgument (metavar "NAME") authtoken = AuthToken <$> strArgument (metavar "TOKEN")- create = Create <$> strArgument (metavar "NAME")- push = Push <$> strArgument (metavar "NAME")+ create = Create <$> nameArg+ generateKeypair = GenerateKeypair <$> nameArg+ push = Push <$> nameArg <*> many (strArgument (metavar "PATHS..."))- <*> switch (long "watch-store" <> short 'w' <> help "Run in daemon mode and push store paths as they are added to /nix/store")- use = Use <$> strArgument (metavar "NAME")- <*> switch (long "nixos" <> short 'n' <> help "Output NixOS configuration lines")+ <*> switch ( long "watch-store"+ <> short 'w'+ <> help "Run in daemon mode and push store paths as they are added to /nix/store"+ )+ use = Use <$> nameArg+ <*> (UseOptions <$> switch ( long "nixos"+ <> short 'n'+ <> help "Write NixOS modules")+ <*> strOption ( long "nixos-folder"+ <> short 'd'+ <> help "Base directory for NixOS configuration"+ <> value "/etc/nixos/"+ <> showDefault+ )) getOpts :: IO (CachixOptions, CachixCommand) getOpts = do
+ test/NetRcSpec.hs view
@@ -0,0 +1,59 @@+module NetRcSpec where++import Paths_cachix+import Protolude+import System.Directory (copyFile)+import System.IO.Temp (withSystemTempFile)+import Test.Hspec++import Cachix.Api (BinaryCache(..))+import qualified Cachix.Client.NetRc as NetRc+import Cachix.Client.Config (Config(..))+++bc1 :: BinaryCache+bc1 = BinaryCache+ { name = "name"+ , uri = "https://name.cachix.org"+ , publicSigningKeys = ["pub"]+ , isPublic = False+ , githubUsername = "foobar"+ }++bc2 :: BinaryCache+bc2 = BinaryCache+ { name = "name2"+ , uri = "https://name2.cachix.org"+ , publicSigningKeys = ["pub2"]+ , isPublic = False+ , githubUsername = "foobar2"+ }++config :: Config+config = Config+ { authToken = "token123"+ , binaryCaches = []+ }++-- TODO: poor man's golden tests, use https://github.com/stackbuilders/hspec-golden+test :: [BinaryCache] -> Text -> Expectation+test binaryCaches goldenName = withSystemTempFile "hspec-netrc" $ \filepath _ -> do+ input <- getInput+ output <- getOutput+ copyFile input filepath+ NetRc.add config binaryCaches filepath+ real <- readFile filepath+ expected <- readFile output+ real `shouldBe` expected+ where+ getInput = getDataFileName $ toS goldenName <> ".input"+ getOutput = getDataFileName $ toS goldenName <> ".output"++spec :: Spec+spec =+ describe "add" $ do+ -- TODO: not easy to test this with temp files as they are *created*+ --it "populates non-existent netrc file" $ test [bc1, bc2] "fresh"+ it "populates empty netrc file" $ test [bc1, bc2] "empty"+ it "populates netrc file with one additional entry" $ test [bc2] "add"+ it "populates netrc file with one overriden entry" $ test [bc2] "override"
test/NixConfSpec.hs view
@@ -20,6 +20,7 @@ bc = BinaryCache { name = "name" , uri = "https://name.cachix.org"+ , isPublic = True , publicSigningKeys = ["pub"] , githubUsername = "foobar" }
+ test/data/add.input view
@@ -0,0 +1,1 @@+machine name.cachix.org password token123
+ test/data/add.output view
@@ -0,0 +1,3 @@+machine name2.cachix.org password token123++machine name.cachix.org password token123
+ test/data/empty.input view
+ test/data/empty.output view
@@ -0,0 +1,3 @@+machine name.cachix.org password token123++machine name2.cachix.org password token123
+ test/data/override.input view
@@ -0,0 +1,3 @@+machine name.cachix.org password token123++machine name2.cachix.org password abc123
+ test/data/override.output view
@@ -0,0 +1,3 @@+machine name2.cachix.org password token123++machine name.cachix.org password token123