elynx-tools 0.0.1 → 0.1.0

raw patch · 12 files changed

+369/−131 lines, 12 filesdep +aesondep +asyncdep +cryptohash-sha256dep ~fast-loggerdep ~zlibPVP ok

version bump matches the API change (PVP)

Dependencies added: aeson, async, cryptohash-sha256, deepseq

Dependency ranges changed: fast-logger, zlib

API changes (from Hackage documentation)

Files

• ChangeLog.md +2/−2
• README.md +88/−78
• elynx-tools.cabal +10/−4
• src/ELynx/Tools/ByteString.hs +3/−3
• src/ELynx/Tools/Concurrent.hs +31/−4
• src/ELynx/Tools/Definitions.hs +5/−0
• src/ELynx/Tools/Equality.hs +12/−1
• src/ELynx/Tools/InputOutput.hs +31/−5
• src/ELynx/Tools/Logger.hs +1/−24
• src/ELynx/Tools/Options.hs +26/−10
• src/ELynx/Tools/Reproduction.hs +125/−0
• src/ELynx/Tools/Text.hs +35/−0

@@ -1,10 +1,10 @@  # Table of Contents -1.  [Changelog for ELynx](#org9916021)+1.  [Changelog for ELynx](#org263dcac)  -<a id="org9916021"></a>+<a id="org263dcac"></a>  # Changelog for ELynx 

@@ -2,33 +2,33 @@  # The ELynx Suite -Version: 0.0.1.+Version: 0.1.0. Reproducible evolution made easy.  The ELynx Suite is a Haskell library and a tool set for computational biology. The goal of the ELynx Suite is reproducible research. Evolutionary sequences and phylogenetic trees can be read, viewed, modified and simulated. Exact specification of all options is necessary, and nothing is assumed about the data-(e.g., the type of code). The command line with all arguments is consistently,-and automatically logged. The work overhead in the beginning usually pays off in-the end.+(e.g., the type of the genetic code). The command line with all arguments is+consistently, and automatically logged. The work overhead in the beginning+usually pays off in the end.  The Elynx Suite consists of three library packages and two executables providing a range of sub commands.  The library packages are: --   **elynx-seq:** Handle evolutionary sequences and multi sequence alignments;--   **elynx-tree:** Handle phylogenetic trees;--   **elynx-tools:** Tools for the provided executables;+-   **elynx-seq:** Handle evolutionary sequences and multi sequence alignments.+-   **elynx-tree:** Handle phylogenetic trees.+-   **elynx-tools:** Tools for the provided executables.  The executables are: --   **SLynx:** Analyze, modify, and simulate evolutionary sequences (FASTA format);--   **TLynx:** Analyze, modify, and simulate phylogenetic trees (Newick format).+-   **slynx:** Analyze, modify, and simulate evolutionary sequences (FASTA format).+-   **tlynx:** Analyze, modify, and simulate phylogenetic trees (Newick format). -**ELynx is still under development. We happily receive comments, ideas, feature-requests, or pull requests!**+**ELynx is actively developed. We happily receive comments, ideas, feature+requests, and pull requests!**   # Installation@@ -57,8 +57,8 @@              stack install     -    The binaries are installed into `~/.local/bin/` which has to be added [PATH](https://en.wikipedia.org/wiki/PATH_(variable)).-    Then, they can be used directly.+    The binaries are installed into `~/.local/bin/` which has to be added to the+    [PATH](https://en.wikipedia.org/wiki/PATH_(variable)) environment variable. Then, they can be used directly.   # SLynx@@ -67,8 +67,8 @@      slynx --help -    ELynx Suite version 0.5.1. Developed by Dominik Schrempf. Compiled on September-    9, 2019, at 10:48 am, UTC.+    ELynx Suite version 0.1.0. Developed by Dominik Schrempf. Compiled on January+    30, 2020, at 09:18 am, UTC.          Usage: slynx [-v|--verbosity VALUE] [-o|--output-file-basename NAME] COMMAND       Analyze, and simulate multi sequence alignments.@@ -82,17 +82,14 @@                                Specify base name of output file          Available commands:-      concatenate              -      examine                  If data is a multi sequence alignment, additionally-                               analyze columns.-      filter-rows              -      filter-columns           -      simulate                 -      sub-sample               Create a given number of multi sequence alignments,-                               each of which containing a given number of random-                               sites drawn from the original multi sequence-                               alignment.-      translate                +      concatenate              Concatenate sequences found in input files.+      examine                  Examine sequences. If data is a multi sequence+                               alignment, additionally analyze columns.+      filter-rows              Filter rows (or sequences) found in input files.+      filter-columns           Filter columns of multi-sequence alignments.+      simulate                 Simulate multi sequence alignments.+      sub-sample               Sub-sample columns from multi sequence alignments.+      translate                Translate from DNA to Protein or DNAX to ProteinX.          File formats:       - FASTA@@ -106,15 +103,20 @@       - ProteinS (amino acids; including gaps, and translation stops)       - ProteinI (amino acids; including gaps, translation stops, and IUPAC codes)     -    The ELynx Suite.+    The ELynx Suite+    ---------------     A Haskell library and a tool set for computational biology. The goal of the     ELynx Suite is reproducible research. Evolutionary sequences and phylogenetic     trees can be read, viewed, modified and simulated. Exact specification of all     options is necessary, and nothing is assumed about the data (e.g., the type of     code). The command line with all arguments is consistently, and automatically-    logged. The work overhead in the beginning usually pays off in the end.+    logged.+         slynx     Analyze, modify, and simulate evolutionary sequences.     tlynx     Analyze, modify, and simulate phylogenetic trees.+    +    Get help for specific commands:+      slynx examine --help   ## Concatenate@@ -123,9 +125,8 @@      slynx concatenate --help -    Concatenate sequences found in input files.-         Usage: slynx concatenate (-a|--alphabet NAME) INPUT-FILE+      Concatenate sequences found in input files.          Available options:       -a,--alphabet NAME       Specify alphabet type NAME@@ -139,10 +140,9 @@      slynx examine --help -    Examine sequences.-         Usage: slynx examine (-a|--alphabet NAME) [INPUT-FILE] [--per-site]-      If data is a multi sequence alignment, additionally analyze columns.+      Examine sequences. If data is a multi sequence alignment, additionally analyze+      columns.          Available options:       -a,--alphabet NAME       Specify alphabet type NAME@@ -157,26 +157,27 @@      slynx filter-rows --help -    Filter rows (or sequences) found in input files.-         Usage: slynx filter-rows (-a|--alphabet NAME) [INPUT-FILE]                              [--longer-than LENGTH] [--shorter-than LENGTH]+                             [--standard-characters]+      Filter rows (or sequences) found in input files.          Available options:       -a,--alphabet NAME       Specify alphabet type NAME       INPUT-FILE               Read sequences from INPUT-FILE       --longer-than LENGTH     Only keep sequences longer than LENGTH       --shorter-than LENGTH    Only keep sequences shorter than LENGTH+      --standard-characters    Only keep sequences containing at least one standard+                               (i.e., non-IUPAC) character       -h,--help                Show this help text  Filter columns of multi sequence alignments with `filter-columns`.      slynx filter-columns --help -    Filter columns of multi-sequence alignments.-         Usage: slynx filter-columns (-a|--alphabet NAME) [INPUT-FILE]                                 [--standard-chars DOUBLE]+      Filter columns of multi-sequence alignments.          Available options:       -a,--alphabet NAME       Specify alphabet type NAME@@ -192,13 +193,13 @@      slynx simulate --help -    Simulate multi sequence alignments.-         Usage: slynx simulate (-t|--tree-file Name) [-s|--substitution-model MODEL]                           [-m|--mixture-model MODEL] [-e|--edm-file NAME]+                          [-p|--siteprofile-files NAMES]                           [-w|--mixture-model-weights "[DOUBLE,DOUBLE,...]"]                           [-g|--gamma-rate-heterogeneity "(NCAT,SHAPE)"]                           (-l|--length NUMBER) [-S|--seed [INT]]+      Simulate multi sequence alignments.          Available options:       -t,--tree-file Name      Read trees from file NAME@@ -210,6 +211,8 @@                                are shown below (mutually exclusive with -s option)       -e,--edm-file NAME       Empirical distribution model file NAME in Phylobayes                                format+      -p,--siteprofile-files NAMES+                               File names of site profiles in Phylobayes format       -w,--mixture-model-weights "[DOUBLE,DOUBLE,...]"                                Weights of mixture model components       -g,--gamma-rate-heterogeneity "(NCAT,SHAPE)"@@ -257,8 +260,7 @@     Usage: slynx sub-sample (-a|--alphabet NAME) [INPUT-FILE]                             (-n|--number-of-sites INT)                             (-m|--number-of-alignments INT) [-S|--seed [INT]]-      Create a given number of multi sequence alignments, each of which containing a-      given number of random sites drawn from the original multi sequence alignment.+      Sub-sample columns from multi sequence alignments.          Available options:       -a,--alphabet NAME       Specify alphabet type NAME@@ -269,6 +271,9 @@       -S,--seed [INT]          Seed for random number generator; list of 32 bit                                integers with up to 256 elements (default: random)       -h,--help                Show this help text+    +    Create a given number of multi sequence alignments, each of which contains a+    given number of random sites drawn from the original multi sequence alignment.   ## Translate@@ -277,10 +282,9 @@      slynx translate --help -    Translate from DNA to Protein or DNAX to ProteinX.-         Usage: slynx translate (-a|--alphabet NAME) [INPUT-FILE]                            (-r|--reading-frame INT) (-u|--universal-code CODE)+      Translate from DNA to Protein or DNAX to ProteinX.          Available options:       -a,--alphabet NAME       Specify alphabet type NAME@@ -297,8 +301,8 @@      tlynx --help -    ELynx Suite version 0.5.1. Developed by Dominik Schrempf. Compiled on September-    9, 2019, at 10:48 am, UTC.+    ELynx Suite version 0.1.0. Developed by Dominik Schrempf. Compiled on January+    30, 2020, at 09:18 am, UTC.          Usage: tlynx [-v|--verbosity VALUE] [-o|--output-file-basename NAME] COMMAND       Compare, examine, and simulate phylogenetic trees.@@ -312,26 +316,34 @@                                Specify base name of output file          Available commands:-      compare                  -      examine                  -      simulate                 Simulate reconstructed trees using the point process.-                               See Gernhard, T. (2008). The conditioned-                               reconstructed process. Journal of Theoretical-                               Biology, 253(4), 769–778.-                               http://doi.org/10.1016/j.jtbi.2008.04.005+      distance                 Compute distances between many phylogenetic trees.+      examine                  Compute summary statistics of phylogenetic trees.+      simulate                 Simulate phylogenetic trees using birth and death+                               processes.+      compare                  Compare two phylogenetic trees (compute distances and+                               branch-wise differences).+      connect                  Connect two phylogenetic trees in all ways (possibly+                               honoring constraints).+      shuffle                  Shuffle a phylogenetic tree (keep coalescent times,+                               but shuffle topology and leaves).          File formats:       - Newick     -    The ELynx Suite.+    The ELynx Suite+    ---------------     A Haskell library and a tool set for computational biology. The goal of the     ELynx Suite is reproducible research. Evolutionary sequences and phylogenetic     trees can be read, viewed, modified and simulated. Exact specification of all     options is necessary, and nothing is assumed about the data (e.g., the type of     code). The command line with all arguments is consistently, and automatically-    logged. The work overhead in the beginning usually pays off in the end.+    logged.+         slynx     Analyze, modify, and simulate evolutionary sequences.     tlynx     Analyze, modify, and simulate phylogenetic trees.+    +    Get help for specific commands:+      slynx examine --help   ## Compare@@ -340,24 +352,18 @@      tlynx compare --help -    Compute distances between phylogenetic trees.-    -    Usage: tlynx compare (-d|--distance MEASURE) [-s|--summary-statistics]-                         [INPUT-FILES]+    Usage: tlynx compare [-n|--normalize] [-b|--bipartitions] [-i|--newick-iqtree]+                         NAME+      Compare two phylogenetic trees (compute distances and branch-wise+      differences).          Available options:-      -d,--distance MEASURE    Type of distance to calculate (available distance-                               measures are listed below)-      -s,--summary-statistics  Report summary statistics only-      INPUT-FILES              Read tree(s) from INPUT-FILES; if more files are-                               given, one tree is expected per file+      -n,--normalize           Normalize trees before comparison+      -b,--bipartitions        Print common and missing bipartitions+      -i,--newick-iqtree       Use IQ-TREE Newick format (internal node labels are+                               branch support values)+      NAME                     Tree file       -h,--help                Show this help text-    -    Available distance measures:-      Symmetric distance: -d symmetric-      Incompatible split distance: -d incompatible-split[VAL]-        Collapse branches with support less than VAL before distance calculation;-        in this way, only well supported difference contribute to the distance measure.   ## Examine@@ -366,12 +372,13 @@      tlynx examine --help -    Compute summary statistics of phylogenetic trees.-    -    Usage: tlynx examine [INPUT-FILE]+    Usage: tlynx examine [INPUT-FILE] [-i|--newick-iqtree]+      Compute summary statistics of phylogenetic trees.          Available options:       INPUT-FILE               Read trees from INPUT-FILE+      -i,--newick-iqtree       Use IQ-TREE Newick format (internal node labels are+                               branch support values)       -h,--help                Show this help text  @@ -381,15 +388,11 @@      tlynx simulate --help -    Simulate phylogenetic trees using birth and death processes.-         Usage: tlynx simulate [-t|--nTrees INT] [-n|--nLeaves INT] [-H|--height DOUBLE]                           [-M|--condition-on-mrca] [-l|--lambda DOUBLE]                           [-m|--mu DOUBLE] [-r|--rho DOUBLE] [-u|--sub-sample]                           [-s|--summary-statistics] [-S|--seed [INT]]-      Simulate reconstructed trees using the point process. See Gernhard, T. (2008).-      The conditioned reconstructed process. Journal of Theoretical Biology, 253(4),-      769–778. http://doi.org/10.1016/j.jtbi.2008.04.005+      Simulate phylogenetic trees using birth and death processes.          Available options:       -t,--nTrees INT          Number of trees (default: 10)@@ -409,9 +412,16 @@     Height of Trees: if no tree height is given, the heights will be randomly drawn from the expected distribution given the number of leaves, the birth and the death rate.     Summary statistics only: only print (NumberOfExtantChildren BranchLength) pairs for each branch of each tree. The trees are separated by a newline character.     Sub-sampling: simulate one big tree with n'=round(n/rho), n'>=n, leaves, and randomly sample sub-trees with n leaves. Hence, with rho=1.0, the same tree is reported over and over again.+    Gernhard, T. (2008). The conditioned reconstructed process. Journal of Theoretical Biology, 253(4), 769–778. http://doi.org/10.1016/j.jtbi.2008.04.005   # ELynx -Documentation of the library can be found on Hackage.+Documentation of the library can be found on [Hackage](https://hackage.haskell.org/):++-   [elynx-seq](https://hackage.haskell.org/package/elynx-seq)+-   [elynx-tree](https://hackage.haskell.org/package/elynx-tree)+-   [elynx-tools](https://hackage.haskell.org/package/elynx-tools)+-   [slynx](https://hackage.haskell.org/package/slynx)+-   [tlynx](https://hackage.haskell.org/package/tlynx) 

@@ -1,6 +1,6 @@ cabal-version: 1.12 name: elynx-tools-version: 0.0.1+version: 0.1.0 license: GPL-3 license-file: LICENSE copyright: Dominik Schrempf (2019)@@ -36,17 +36,23 @@         ELynx.Tools.Numeric         ELynx.Tools.Options         ELynx.Tools.Random+        ELynx.Tools.Reproduction+        ELynx.Tools.Text         ELynx.Tools.Vector     hs-source-dirs: src     other-modules:         Paths_elynx_tools     default-language: Haskell2010-    ghc-options: -Wall+    ghc-options: -Wall -O2     build-depends:+        aeson >=1.4.5.0 && <1.5,+        async >=2.2.2 && <2.3,         base >=4.12.0.0 && <4.13,         bytestring >=0.10.8.2 && <0.11,         containers >=0.6.0.1 && <0.7,-        fast-logger >=2.4.16 && <2.5,+        cryptohash-sha256 >=0.11.101.0 && <0.12,+        deepseq >=1.4.4.0 && <1.5,+        fast-logger >=2.4.17 && <2.5,         hmatrix >=0.20.0.0 && <0.21,         lifted-base >=0.2.3.12 && <0.3,         matrices >=0.5.0 && <0.6,@@ -63,4 +69,4 @@         transformers >=0.5.6.2 && <0.6,         transformers-base >=0.4.5.2 && <0.5,         vector >=0.12.0.3 && <0.13,-        zlib >=0.6.2 && <0.7+        zlib >=0.6.2.1 && <0.7

@@ -23,7 +23,7 @@   , summarizeByteString   , c2w   , w2c-  , bsShow+  , bShow   ) where  import           Data.ByteString.Internal   (c2w, w2c)@@ -58,5 +58,5 @@                         | otherwise = s  -- | Conversion to 'L.ByteString'.-bsShow :: Show a => a -> L.ByteString-bsShow = L.pack . show+bShow :: Show a => a -> L.ByteString+bShow = L.pack . show

@@ -1,3 +1,4 @@+{-# LANGUAGE ScopedTypeVariables #-} {- | Module      :  ELynx.Tools.Concurrent Description :  Tools for concurrent random calculations@@ -18,16 +19,23 @@     getNGen   , splitGen    -- * Parallel stuff+  , parComp   , getChunks   , parMapChunk   ) where +import           Control.Concurrent+import           Control.Concurrent.Async import           Control.Monad+-- import qualified Control.Monad.Parallel      as P+import           Control.Monad.Primitive import           Control.Parallel.Strategies import qualified Data.Vector                 as V import           Data.Word import           System.Random.MWC +-- import           ELynx.Tools.Definitions+ -- | Get a number of generators, possibly with a fixed seed. getNGen :: Int -> Maybe [Word32] -> IO [GenIO] getNGen n ms = do@@ -35,13 +43,31 @@   splitGen n g  -- | Split a generator.-splitGen :: Int -> GenIO -> IO [GenIO]+splitGen :: PrimMonad m => Int -> Gen (PrimState m) -> m [Gen (PrimState m)] splitGen n gen   | n <= 0    = return []-  | otherwise =-  fmap (gen:) . replicateM (n-1) $-  initialize =<< (uniformVector gen 256 :: IO (V.Vector Word32))+  | otherwise = do+      seeds :: [V.Vector Word32] <- replicateM n $ uniformVector gen 256+      mapM initialize seeds +-- -- XXX: This just doesn't work... The only thing I found:+-- -- https://stackoverflow.com/a/16250010.+-- parComp :: (PrimMonad m, Monoid b) => Int -> (Int -> Gen (PrimState m) -> m b)+--         -> Gen (PrimState m) -> m b+-- parComp num fun gen = do+--   let ncap   = ceiling (fromIntegral num / fromIntegral chunksize :: Double)+--       chunks = getChunks ncap num+--   gs <- splitGen ncap gen+--   mconcat <$> P.mapM (\(n', g') -> fun n' g') (zip chunks gs)++-- | Perform random calculation in parallel. Does only work with 'IO' and the moment.+parComp :: Int -> (Int -> GenIO -> IO b) -> GenIO -> IO [b]+parComp num fun gen = do+  ncap   <- getNumCapabilities+  let chunks = getChunks ncap num+  gs <- splitGen ncap gen+  mapConcurrently (uncurry fun) (zip chunks gs)+ -- | For a given number of capabilities and number of calculations, get chunk -- sizes. The chunk sizes will be as evenly distributed as possible and sum up -- to the number of calculations.@@ -51,6 +77,7 @@         r = n `mod` c         ns = replicate r (n'+1) ++ replicate (c - r) n' +-- XXX: This should actually be in a module called Parallel, not Concurrent. -- | Parallel map with given chunk size. parMapChunk :: Int -> (a -> b) -> [a] -> [b] parMapChunk n f as = map f as `using` parListChunk n rseq

@@ -15,6 +15,7 @@ module ELynx.Tools.Definitions   ( eps   , precision+  , chunksize   ) where  -- | Required precision when comparing 'Double' values.@@ -24,3 +25,7 @@ -- | Default output precision. precision :: Int precision = 3++-- | Default chunk size for parallel evaluations.+chunksize :: Int+chunksize = 500

@@ -16,6 +16,8 @@ module ELynx.Tools.Equality   (     allEqual+  , allNearlyEqualWith+  , allNearlyEqual   , nearlyEqWith   , eps   , nearlyEq@@ -39,6 +41,15 @@ allEqual [] = True allEqual xs = all (== head xs) (tail xs) +-- | Test if all elements of a list are nearly equal; returns True for empty list.+allNearlyEqualWith :: Double -> [Double] -> Bool+allNearlyEqualWith _   [] = True+allNearlyEqualWith tol xs = all (nearlyEqWith tol $ head xs) (tail xs)++-- | Test if all elements of a list are nearly equal; returns True for empty list.+allNearlyEqual :: [Double] -> Bool+allNearlyEqual = allNearlyEqualWith eps+ -- | Test for equality with given tolerance (needed because of machine precision). nearlyEqWith :: Double -> Double -> Double -> Bool nearlyEqWith tol a b = tol > abs (a-b)@@ -52,7 +63,7 @@ (=~=) :: Double -> Double -> Bool (=~=) = nearlyEq --- Test if the given number is nearly equal to all elements of a vector.+-- Test if the given number is nearly equal to all elements of a list. nearlyEqValListWith :: Double -> Double -> [Double] -> Bool nearlyEqValListWith tol a = all (nearlyEqWith tol a) 

@@ -21,7 +21,8 @@     -- * Input, output.     readGZFile   , writeGZFile-  , io+  , out+  , outHandle     -- * Parsing.   , runParserOnFile   , parseFileWith@@ -32,19 +33,32 @@   ) where  import           Codec.Compression.GZip     (compress, decompress)+import           Control.DeepSeq            (force)+import           Control.Exception          (evaluate)+import           Control.Monad              ((<=<)) import           Control.Monad.IO.Class import           Control.Monad.Logger import qualified Data.ByteString.Lazy.Char8 as L import           Data.List                  (isSuffixOf) import           Data.Maybe import qualified Data.Text                  as T+import           System.IO import           Text.Megaparsec +-- XXX: For now, all files are read strictly (see help of+-- Control.DeepSeq.force).+readFile' :: FilePath -> IO L.ByteString+-- -- Doesn't work.+-- readFile' fn = withFile fn ReadMode $ (evaluate . force) <=< L.hGetContents+-- This works!+readFile' fn = withFile fn ReadMode $+  (evaluate . force) <=< L.hGetContents+ -- | Read file. If file path ends with ".gz", assume gzipped file and decompress -- before read. readGZFile :: FilePath -> IO L.ByteString-readGZFile f | ".gz" `isSuffixOf` f = decompress <$> L.readFile f-             | otherwise            = L.readFile f+readGZFile f | ".gz" `isSuffixOf` f = decompress <$> readFile' f+             | otherwise            = readFile' f  -- | Write file. If file path ends with ".gz", assume gzipped file and compress -- before write.@@ -100,8 +114,8 @@                             Right val -> val  -- | Write a result with a given name to file or standard output.-io :: (MonadLogger m, MonadIO m) => String -> L.ByteString -> Maybe FilePath -> m ()-io name res mfp =+out :: (MonadLogger m, MonadIO m) => String -> L.ByteString -> Maybe FilePath -> m ()+out name res mfp =   case mfp of     Nothing -> do       $(logInfo) $ T.pack $ "Write " <> name <> " to standard output."@@ -109,3 +123,15 @@     Just fp -> do       $(logInfo) $ T.pack $ "Write " <> name <> " to file '" <> fp <> "'."       liftIO $ writeGZFile fp res++-- | Get an output handle, does not support compression. The handle has to be+-- closed after use!+outHandle :: (MonadLogger m, MonadIO m) => String -> Maybe FilePath -> m Handle+outHandle name mfp =+  case mfp of+    Nothing -> do+      $(logInfo) $ T.pack $ "Write " <> name <> " to standard output."+      return stdout+    Just fp -> do+      $(logInfo) $ T.pack $ "Write " <> name <> " to file '" <> fp <> "'."+      liftIO $ openFile fp WriteMode

@@ -28,7 +28,6 @@ import           Control.Monad.Trans.Control import qualified Data.ByteString.Char8       as B import           Data.Text--- import qualified Data.Text                   as T import           System.IO import           System.Log.FastLogger @@ -36,9 +35,7 @@  -- | Unified way of creating a new section in the log. logNewSection :: MonadLogger m => Text -> m ()-logNewSection s = do-  $(logInfo) ""-  $(logInfo) $ "-- " <> s+logNewSection s = $(logInfo) $ "-- " <> s  -- | The 'LoggingT' wrapper for ELynx. Prints a header and a footer, logs to -- 'stderr' if no file is provided. If a log file is provided, log to the file@@ -97,24 +94,4 @@            -> LogLevel            -> LogStr            -> B.ByteString--- logStrToBS loc src lvl msg =---   fromLogStr $ getLogStr loc src lvl msg logStrToBS _ _ _ msg = fromLogStr msg <> "\n"---- getLogStr :: Loc---           -> LogSource---           -> LogLevel---           -> LogStr---           -> LogStr--- getLogStr _ src level msg =---     "[" `mappend` logLevelStr level `mappend`---     (if T.null src---         then mempty---         else "#" `mappend` toLogStr src) `mappend`---     "] " `mappend`---     msg `mappend` "\n"---- logLevelStr :: LogLevel -> LogStr--- logLevelStr level = case level of---     LevelOther t -> toLogStr t---     _            -> toLogStr $ B.pack $ drop 5 $ show level

@@ -22,6 +22,7 @@     -- * Options   , parseArgumentsWith   , Verbosity (..)+  , Arguments (..)   , GlobalArguments (..)   , globalArguments   , seedOpt@@ -78,19 +79,23 @@   t  <- time   p  <- getProgName   as <- getArgs-  let l = length desc+  -- let l = length desc   return $ intercalate "\n"-    [ replicate (l+3) '-'-    , "-- " <> desc+    -- [ replicate (l+4) '-'+    [ "=== " <> desc     , hdr-    , "Time: " ++ t+    , "Start time: " ++ t     , "Command line: " ++ p ++ " " ++ unwords as ]  -- | See 'logHeader' but footer. logFooter :: IO String logFooter = do   t <- time-  return $ "Time: " ++ t+  let timeStr = "=== End time: " ++ t+      -- l       = length timeStr+  return $ intercalate "\n"+    [ timeStr ]+    -- , replicate l '-' ]  versionOpt :: Parser (a -> a) versionOpt = infoOption hdr@@ -104,24 +109,30 @@ evoModSuiteFooter =   [ empty   -- , bold $ text "The ELynx Suite."-  , text "The ELynx Suite."-  , fillParagraph "A Haskell library and a tool set for computational biology. The goal of the ELynx Suite is reproducible research. Evolutionary sequences and phylogenetic trees can be read, viewed, modified and simulated. Exact specification of all options is necessary, and nothing is assumed about the data (e.g., the type of code). The command line with all arguments is consistently, and automatically logged. The work overhead in the beginning usually pays off in the end."+  , text "The ELynx Suite"+  , text "---------------"+  , fillParagraph "A Haskell library and a tool set for computational biology. The goal of the ELynx Suite is reproducible research. Evolutionary sequences and phylogenetic trees can be read, viewed, modified and simulated. Exact specification of all options is necessary, and nothing is assumed about the data (e.g., the type of code). The command line with all arguments is consistently, and automatically logged."   -- , fill 9 (bold $ text "slynx") <+> text "Analyze, modify, and simulate evolutionary sequences."   -- , fill 9 (bold $ text "tlynx") <+> text "Analyze, modify, and simulate phylogenetic trees." ]+  , empty   , fill 9 (text "slynx") <+> text "Analyze, modify, and simulate evolutionary sequences."-  , fill 9 (text "tlynx") <+> text "Analyze, modify, and simulate phylogenetic trees." ]+  , fill 9 (text "tlynx") <+> text "Analyze, modify, and simulate phylogenetic trees."+  , empty+  , text "Get help for specific commands:"+  , text "  slynx examine --help"]  -- | Parse arguments. Provide a global description, header, footer, and so on. -- Custom additional description (first argument) and footer (second argument) -- can be provided. print help if needed.-parseArgumentsWith :: [String] -> [String] -> Parser a -> IO a+parseArgumentsWith :: [String] -> [String] -> Parser a -> IO (Arguments a) parseArgumentsWith desc ftr p = execParser $-  info (helper <*> versionOpt <*> p)+  info (helper <*> versionOpt <*> p')   (fullDesc     <> header hdr     <> progDesc dsc'     <> footerDoc (Just ftr'))   where+    p'   = Arguments <$> globalArguments <*> p     dsc' = unlines desc     ftr' = vsep $ map pretty ftr ++ evoModSuiteFooter @@ -134,6 +145,11 @@ toLogLevel Warning = LevelWarn toLogLevel Info    = LevelInfo toLogLevel Debug   = LevelDebug++-- | Argument skeleton to be used with all commands.+data Arguments a = Arguments { global :: GlobalArguments+                             , local  :: a+                             }  -- | A set of global arguments used by all programs. The idea is to provide a -- common framework for shared arguments.

@@ -0,0 +1,125 @@+{-# LANGUAGE DeriveGeneric       #-}+{-# LANGUAGE ScopedTypeVariables #-}++{- |+Module      :  ELynx.Tools.Reproduction+Description :  Functions to ease reproduction of analyses+Copyright   :  (c) Dominik Schrempf 2019+License     :  GPL-3++Maintainer  :  dominik.schrempf@gmail.com+Stability   :  unstable+Portability :  portable++Creation date: Tue Nov 19 15:07:09 2019.++XXX: This module is actively developed. It is not yet used by slynx nor tlynx.++-}++module ELynx.Tools.Reproduction+  ( Reproducible (..)+  , Reproduction (..)+  , readR+  , writeR+  ) where++import           Control.Monad         (zipWithM)+import           Crypto.Hash.SHA256    (hash)+import           Data.Aeson            (FromJSON, ToJSON,+                                        eitherDecodeFileStrict', encodeFile)+import           Data.Bifunctor        (first)+import qualified Data.ByteString.Char8 as B+import           Data.Either           (either)+import           GHC.Generics          (Generic)+import           Options.Applicative   (Parser, briefDesc, defaultPrefs,+                                        execParserPure, getParseResult, info)+import           System.Environment    (getArgs, getProgName)++-- | Reproducible commands have a set of input files that have to be checked for+-- consistency.+class Reproducible a where+  inFiles :: a -> [FilePath]+  parser  :: a -> Parser a++-- | Necessary information for a reproducible run. Notably, the input files are+-- checked for consistency!+data Reproduction a = Reproduction+  { progName  :: String         -- ^ Program name.+  , args      :: [String]       -- ^ Command line arguments without program name.+  , filePaths :: [FilePath]     -- ^ File paths of in files.+  , checkSums :: [String]       -- ^ SHA256 sums of in files.+  , cmd       :: a              -- ^ Command argument.+  } deriving (Generic)++instance ToJSON a => ToJSON (Reproduction a) where++instance FromJSON a => FromJSON (Reproduction a)++-- Does the command line fit the provided command?+checkArgs :: (Eq a, Show a, Reproducible a)+          => [String] -> a -> IO (Either String ())+checkArgs as c = do+  let p    = parser c+      pres = execParserPure defaultPrefs (info p briefDesc) as+  return $ case getParseResult pres of+    Nothing  ->+      Left $ unlines [ "Could not parse command line string:"+                     , concat as ]+    Just c'  ->+      if c' /= c+      then Left $ unlines [ "Command line string and command arguments do not fit:"+                          , concat as+                          , show c ]+      else Right ()++-- Does the file match the checksum?+checkFile :: FilePath -> B.ByteString -> IO (Either String ())+checkFile fp h = do+  h' <- hashFile fp+  return $ if h' == h+    then Right ()+    else Left $ unlines [ "SHA256 sum does not match for a file."+                        , fp ++ " has check sum " ++ B.unpack h'+                        , "Stored sum is " ++ B.unpack h ]++-- | Check if command line arguments and files check sums are matching.+checkReproduction :: (Eq a, Show a, Reproducible a)+                  => Reproduction a -> IO (Either String ())+checkReproduction (Reproduction _ as fs ss c) = do+  chA  <- checkArgs as c+  chFs <- zipWithM checkFile fs (map B.pack ss)+  let ch = sequence_ (chA : chFs)+  return $ first ("Failed validating the reproduction file.\n" ++) ch++-- | Read an ELynx reproduction file. Check consistency of arguments and input files.+readR :: forall a . (Eq a, Show a, Reproducible a, FromJSON a)+      => FilePath -> IO (Reproduction a)+readR fp = do+  res <- eitherDecodeFileStrict' fp :: IO (Either String (Reproduction a))+  case res of+    Left err -> do+      putStrLn "Failed reading the ELynx reproduction file."+      putStrLn "The following error was encountered."+      error err+    Right r  -> do+      ch <- checkReproduction r+      return $ either error (const r) ch++-- | Helper function.+hashFile :: FilePath -> IO B.ByteString+hashFile f = hash <$> B.readFile f++-- | Write an ELynx reproduction file. Check arguments.+writeR :: (Eq a, Show a, Reproducible a, ToJSON a) => FilePath -> a -> IO ()+writeR fp c = do+  p  <- getProgName+  as <- getArgs+  let fs = inFiles c+  cs <- mapM hashFile fs+  let+    cs' = map B.unpack cs+    r   = Reproduction p as fs cs' c+  -- XXX: Actually, it is only necessary to to checkArgs here. But let's just be safe.+  ch <- checkReproduction r+  either error (const $ encodeFile fp r) ch

@@ -0,0 +1,35 @@+{- |+Module      :  ELynx.Tools.Text+Copyright   :  (c) Dominik Schrempf 2019+License     :  GPL-3++Maintainer  :  dominik.schrempf@gmail.com+Stability   :  unstable+Portability :  portable++Creation date: Thu Feb 14 13:24:53 2019.++indispensable tools for ByteString handling :).++-}++module ELynx.Tools.Text+  (+    -- * Text handling.+    tShow+  , fromBs+  ) where++import           Data.ByteString.Lazy    (ByteString)+import           Data.Text               (Text, pack)+import           Data.Text.Lazy          (toStrict)+import           Data.Text.Lazy.Encoding (decodeUtf8)++-- | Conversion to 'Text' from showable values.+tShow :: Show a => a -> Text+tShow = pack . show++-- | Conversion to 'Text' from bytestring.+fromBs :: ByteString -> Text+fromBs = toStrict . decodeUtf8+