diff --git a/LICENSE b/LICENSE new file mode 100644 --- /dev/null +++ b/LICENSE @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/Setup.hs b/Setup.hs new file mode 100644 --- /dev/null +++ b/Setup.hs @@ -0,0 +1,2 @@ +import Distribution.Simple +main = defaultMain diff --git a/hadoop-tools.cabal b/hadoop-tools.cabal new file mode 100644 --- /dev/null +++ b/hadoop-tools.cabal @@ -0,0 +1,45 @@ +name: hadoop-tools +version: 0.2 +synopsis: Tools for working with Hadoop. +description: Tools for working with Hadoop. +license: Apache-2.0 +license-file: LICENSE +author: Jacob Stanley, Conrad Parker +maintainer: Jacob Stanley +category: Data +build-type: Simple +cabal-version: >= 1.10 + +executable hh + default-language: Haskell2010 + + main-is: Main.hs + hs-source-dirs: src + + other-modules: + Paths_hadoop_tools + + build-depends: + base >= 4.7 && < 5 + , boxes >= 0.1 + , bytestring >= 0.10 + , cereal >= 0.4 + , cereal-conduit >= 0.7 + , conduit >= 1.2 + , conduit-extra >= 1.1 + , configurator >= 0.3 + , directory >= 1.2 + , filepath >= 1.3 + , network >= 2.5 + , network-simple >= 0.4 + , old-locale >= 1.0 + , optparse-applicative >= 0.10 + , protobuf >= 0.2.0.4 + , socks >= 0.5 + , split >= 0.2 + , text >= 1.1 + , time >= 1.4 + , transformers >= 0.4 + , unix >= 2.7 + , unordered-containers >= 0.2 + , xmlhtml >= 0.2.3.2 diff --git a/src/Main.hs b/src/Main.hs new file mode 100644 --- /dev/null +++ b/src/Main.hs @@ -0,0 +1,436 @@ +{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE RecordWildCards #-} + +module Main (main) where + +import Control.Exception (SomeException, handle, throwIO) +import Control.Monad +import Control.Monad.IO.Class (MonadIO, liftIO) + +import Data.Bits ((.&.), shiftR) +import qualified Data.ByteString.Char8 as B +import Data.Foldable (foldMap) +import qualified Data.HashMap.Lazy as H +import Data.List (intercalate) +import Data.List (isPrefixOf) +import Data.List.Split (splitOn) +import Data.Maybe (fromMaybe, maybeToList, mapMaybe, listToMaybe) +import Data.Text (Text) +import qualified Data.Text as T +import qualified Data.Text.Encoding as T +import qualified Data.Text.IO as T +import qualified Data.Text.Read as T +import Data.Time +import Data.Time.Clock.POSIX +import Data.Word (Word16, Word32, Word64) + +import qualified Data.Configurator as C +import Data.Configurator.Types (Worth(..)) +import System.Environment (getEnv, lookupEnv) +import System.FilePath.Posix +import System.IO.Unsafe (unsafePerformIO) +import System.Locale (defaultTimeLocale) +import System.Posix.User (getEffectiveUserName) +import Text.PrettyPrint.Boxes hiding ((<>)) +import Text.XmlHtml + +import Data.Conduit (handleC) +import Data.ProtocolBuffers +import Data.ProtocolBuffers.Orphans () + +import Hadoop.Protobuf.Hdfs +import Hadoop.Rpc + +import Options.Applicative hiding (Success) + +import Data.Version (showVersion) +import Paths_hadoop_tools (version) + +------------------------------------------------------------------------ + +main :: IO () +main = do + cmd <- execParser optsParser + handle printError (runRemote cmd) + where + optsParser = info (helper <*> options) + (fullDesc <> header "hh - Blazing fast interaction with HDFS") + +runRemote :: Remote a -> IO a +runRemote remote = do + hdfsUser <- getHdfsUser + nameNode <- getNameNode + socksProxy <- getSocksProxy + + let run = maybe runTcp runSocks socksProxy + run nameNode hdfsUser remote + +------------------------------------------------------------------------ + +configPath :: FilePath +configPath = unsafePerformIO $ do + home <- getEnv "HOME" + return (home ".hh") +{-# NOINLINE configPath #-} + +getHdfsUser :: IO User +getHdfsUser = attempt [fromEnv, fromCfg] fromUnix + where + fromEnv :: IO (Maybe User) + fromEnv = fmap T.pack <$> lookupEnv "HADOOP_USER_NAME" + + fromCfg :: IO (Maybe User) + fromCfg = C.load [Optional configPath] >>= flip C.lookup "hdfs.user" + + fromUnix :: IO User + fromUnix = T.pack <$> getEffectiveUserName + +getNameNode :: IO NameNode +getNameNode = attempt [fromHadoopCfg, fromUserCfg] explode + where + fromHadoopCfg :: IO (Maybe NameNode) + fromHadoopCfg = listToMaybe . maybe [] id <$> readHadoopNameNodes + + fromUserCfg :: IO (Maybe NameNode) + fromUserCfg = do + cfg <- C.load [Optional configPath] + host <- C.lookup cfg "namenode.host" + port <- C.lookupDefault 8020 cfg "namenode.port" + return (Endpoint <$> host <*> pure port) + + explode :: IO a + explode = error $ "could not find namenode details in /etc/hadoop/conf/ or " <> configPath + +getSocksProxy :: IO (Maybe SocksProxy) +getSocksProxy = do + cfg <- C.load [Optional configPath] + mhost <- C.lookup cfg "proxy.host" + case mhost of + Nothing -> return Nothing + Just host -> Just . Endpoint host <$> C.lookupDefault 1080 cfg "proxy.port" + +attempt :: Monad m => [m (Maybe a)] -> m a -> m a +attempt [] def = def +attempt (io:ios) def = io >>= \mx -> case mx of + Just x -> return x + Nothing -> attempt ios def + +------------------------------------------------------------------------ + +type HadoopConfig = H.HashMap Text Text + +readHadoopNameNodes :: IO (Maybe [NameNode]) +readHadoopNameNodes = do + cfg <- H.union <$> readHadoopConfig "/etc/hadoop/conf/core-site.xml" + <*> readHadoopConfig "/etc/hadoop/conf/hdfs-site.xml" + return $ resolveNameNode cfg <$> (stripProto =<< H.lookup fsDefaultNameKey cfg) + where + proto = "hdfs://" + fsDefaultNameKey = "fs.defaultFS" + nameNodesPrefix = "dfs.ha.namenodes." + rpcAddressPrefix = "dfs.namenode.rpc-address." + + stripProto :: Text -> Maybe Text + stripProto uri | proto `T.isPrefixOf` uri = Just (T.drop (T.length proto) uri) + | otherwise = Nothing + + resolveNameNode :: HadoopConfig -> Text -> [NameNode] + resolveNameNode cfg name = case parseEndpoint name of + Just ep -> [ep] -- contains "host:port" directly + Nothing -> mapMaybe (\nn -> lookupAddress cfg $ name <> "." <> nn) + (lookupNameNodes cfg name) + + lookupNameNodes :: HadoopConfig -> Text -> [Text] + lookupNameNodes cfg name = maybe [] id + $ T.splitOn "," <$> H.lookup (nameNodesPrefix <> name) cfg + + lookupAddress :: HadoopConfig -> Text -> Maybe Endpoint + lookupAddress cfg name = parseEndpoint =<< H.lookup (rpcAddressPrefix <> name) cfg + + parseEndpoint :: Text -> Maybe Endpoint + parseEndpoint ep = Endpoint host <$> port + where + host = T.takeWhile (/= ':') ep + port = either (const Nothing) (Just . fst) + $ T.decimal $ T.drop (T.length host + 1) ep + +readHadoopConfig :: FilePath -> IO HadoopConfig +readHadoopConfig path = do + exml <- parseXML path <$> B.readFile path + case exml of + Left _ -> return H.empty + Right xml -> return (toHashMap (docContent xml)) + where + toHashMap = H.fromList . mapMaybe fromNode + . concatMap (descendantElementsTag "property") + + fromNode n = (,) <$> (nodeText <$> childElementTag "name" n) + <*> (nodeText <$> childElementTag "value" n) + +------------------------------------------------------------------------ + +workingDirConfigPath :: FilePath +workingDirConfigPath = unsafePerformIO $ do + home <- getEnv "HOME" + return (home ".hhwd") +{-# NOINLINE workingDirConfigPath #-} + +getDefaultWorkingDir :: MonadIO m => m FilePath +getDefaultWorkingDir = liftIO $ (("/user" ) . T.unpack) <$> getHdfsUser + +getWorkingDir :: MonadIO m => m FilePath +getWorkingDir = liftIO $ handle onError + $ T.unpack . T.takeWhile (/= '\n') + <$> T.readFile workingDirConfigPath + where + onError :: SomeException -> IO FilePath + onError = const getDefaultWorkingDir + +setWorkingDir :: MonadIO m => FilePath -> m () +setWorkingDir path = liftIO $ T.writeFile workingDirConfigPath + $ T.pack path <> "\n" + +getAbsolute :: MonadIO m => FilePath -> m FilePath +getAbsolute path = liftIO (normalizePath <$> getPath) + where + getPath = if "/" `isPrefixOf` path + then return path + else getWorkingDir >>= \pwd -> return (pwd path) + +normalizePath :: FilePath -> FilePath +normalizePath = intercalate "/" . dropAbsParentDir . splitOn "/" + +dropAbsParentDir :: [FilePath] -> [FilePath] +dropAbsParentDir [] = error "dropAbsParentDir: not an absolute path" +dropAbsParentDir (p : ps) = p : (reverse $ fst $ go [] ps) + where + go [] (".." : ys) = go [] ys + go (_ : xs) (".." : ys) = go xs ys + go xs (y : ys) = go (y : xs) ys + go xs [] = (xs, []) + +------------------------------------------------------------------------ + +data SubCommand = SubCommand + { subName :: String + , subDescription :: String + , subMethod :: Parser (Remote ()) + } + +sub :: SubCommand -> Mod CommandFields (Remote ()) +sub SubCommand{..} = command subName (info subMethod $ progDesc subDescription) + +options :: Parser (Remote ()) +options = subparser (foldMap sub allSubCommands) + +allSubCommands :: [SubCommand] +allSubCommands = + [ subChDir + , subDiskUsage + , subList + , subMkDir + , subPwd + , subRemove + , subRename + , subVersion + ] + +completePath :: Mod ArgumentFields a +completePath = completer (fileCompletion (const True)) <> metavar "PATH" + +completeDir :: Mod ArgumentFields a +completeDir = completer (fileCompletion (== Dir)) <> metavar "DIRECTORY" + +subChDir :: SubCommand +subChDir = SubCommand "cd" "Change working directory" go + where + go = cd <$> optional (argument str (completeDir <> help "the directory to change to")) + cd mpath = do + path <- getAbsolute =<< maybe getDefaultWorkingDir return mpath + mls <- getListing path + liftIO $ case mls of + Nothing -> putStrLn $ "Directory does not exist: " <> path + Just _ -> setWorkingDir path + +subDiskUsage :: SubCommand +subDiskUsage = SubCommand "du" "Show the amount of space used by file or directory" go + where + go = du <$> optional (argument str (completePath <> help "the file/directory to check the usage of")) + du path = printDiskUsage =<< getAbsolute (maybe "" id path) + +subList :: SubCommand +subList = SubCommand "ls" "List the contents of a directory" go + where + go = ls <$> (optional (argument str (completeDir <> help "the directory to list"))) + ls path = printListing =<< getAbsolute (fromMaybe "" path) + +subMkDir :: SubCommand +subMkDir = SubCommand "mkdir" "Create a directory in the specified location" go + where + go = mkdir <$> argument str (completeDir <> help "the directory to create") + <*> switch (short 'p' <> help "create intermediate directories") + mkdir path parent = do + absPath <- getAbsolute path + ok <- mkdirs absPath parent + unless ok $ liftIO . putStrLn $ "Failed to create: " <> absPath + +subPwd :: SubCommand +subPwd = SubCommand "pwd" "Print working directory" go + where + go = pure pwd + pwd = liftIO . putStrLn =<< getWorkingDir + +subRemove :: SubCommand +subRemove = SubCommand "rm" "Delete a file or directory" go + where + go = rm <$> argument str (completePath <> help "the file/directory to remove") + <*> switch (short 'r' <> help "recursively remove the whole file hierarchy") + rm path recursive = do + absPath <- getAbsolute path + ok <- delete absPath recursive + unless ok $ liftIO . putStrLn $ "Failed to remove: " <> absPath + +subRename :: SubCommand +subRename = SubCommand "mv" "Rename a file or directory" go + where + go = mv <$> argument str (completePath <> help "source file/directory") + <*> argument str (completePath <> help "destination file/directory") + <*> switch (short 'f' <> help "overwrite destination if it exists") + mv src dst force = do + absSrc <- getAbsolute src + absDst <- getAbsolute dst + rename absSrc absDst force + +subVersion :: SubCommand +subVersion = SubCommand "version" "Show version information" go + where + go = pure $ liftIO $ putStrLn $ "hh version " <> showVersion version + +------------------------------------------------------------------------ + +fileCompletion :: (FileType -> Bool) -> Completer +fileCompletion p = mkCompleter $ \path -> handle ignore $ runRemote $ do + let (dir, _) = splitFileName' path + ls <- getListing =<< getAbsolute dir + + return $ filter (path `isPrefixOf`) + . map (displayPath dir) + . filter (p . get fsFileType) + . concatMap (get dlPartialListing) + . maybeToList $ ls + where + ignore (RemoteError _ _) = return [] + + splitFileName' x = case splitFileName x of + ("./", f) -> ("", f) + (d, f) -> (d, f) + +displayPath :: FilePath -> FileStatus -> FilePath +displayPath parent file = parent B.unpack (get fsPath file) <> suffix + where + suffix = case get fsFileType file of + Dir -> "/" + _ -> "" + +------------------------------------------------------------------------ + +printDiskUsage :: FilePath -> Remote () +printDiskUsage path = do + mls <- getListing path + case mls of + Nothing -> liftIO $ putStrLn $ "File/directory does not exist: " <> path + Just ls -> do + let files = map (displayPath path) (get dlPartialListing ls) + css <- zip files <$> mapM getDirSize files + + let col a f = vcat a (map (text . f) css) + + liftIO $ do + printBox $ col right snd + <+> col left fst + where + getDirSize f = handleC (\e -> if isAccessDenied e then return "-" else liftIO (throwIO e)) + (formatSize . get csLength <$> getContentSummary f) + +printListing :: FilePath -> Remote () +printListing path = do + mls <- getListing path + case mls of + Nothing -> liftIO . putStrLn $ "Directory does not exist: " <> path + Just ls -> do + let getPerms = fromIntegral . get fpPerm . get fsPermission + getBlockRepl = fromMaybe 0 . get fsBlockReplication + + hdfs2utc ms = posixSecondsToUTCTime (fromIntegral ms / 1000) + getModTime = hdfs2utc . get fsModificationTime + + -- TODO: Fetch rest of partial listing + xs = get dlPartialListing ls + col a f = vcat a (map (text . f) xs) + + liftIO $ do + putStrLn $ "Found " <> show (length xs) <> " items" + + printBox $ col left (\x -> formatMode (get fsFileType x) (getPerms x)) + <+> col right (formatBlockRepl . getBlockRepl) + <+> col left (T.unpack . get fsOwner) + <+> col left (T.unpack . get fsGroup) + <+> col right (formatSize . get fsLength) + <+> col right (formatUTC . getModTime) + <+> col left (T.unpack . T.decodeUtf8 . get fsPath) + +------------------------------------------------------------------------ + +get :: HasField a => (t -> a) -> t -> FieldType a +get f x = getField (f x) + +------------------------------------------------------------------------ + +type Perms = Word16 + +formatSize :: Word64 -> String +formatSize b | b <= 0 = "0" + | b < 1000 = show b <> "B" + | b < 1000000 = show (b `div` 1000) <> "K" + | b < 1000000000 = show (b `div` 1000000) <> "M" + | b < 1000000000000 = show (b `div` 1000000000) <> "G" + | otherwise = show (b `div` 1000000000000) <> "T" + +formatBlockRepl :: Word32 -> String +formatBlockRepl x | x == 0 = "-" + | otherwise = show x + +formatUTC :: UTCTime -> String +formatUTC = formatTime defaultTimeLocale "%Y-%m-%d %H:%M" + +formatMode :: FileType -> Perms -> String +formatMode File = ("-" <>) . formatPerms +formatMode Dir = ("d" <>) . formatPerms +formatMode SymLink = ("l" <>) . formatPerms + +formatPerms :: Perms -> String +formatPerms perms = format (perms `shiftR` 6) + <> format (perms `shiftR` 3) + <> format perms + where + format p = conv 0x4 "r" p + <> conv 0x2 "w" p + <> conv 0x1 "x" p + + conv bit rep p | (p .&. bit) /= 0 = rep + | otherwise = "-" + +------------------------------------------------------------------------ + +printError :: RemoteError -> IO () +printError (RemoteError subject body) + | oneLiner = T.putStrLn firstLine + | otherwise = T.putStrLn subject >> T.putStrLn body + where + oneLiner = subject `elem` [ "org.apache.hadoop.security.AccessControlException" + , "org.apache.hadoop.fs.FileAlreadyExistsException" + , "java.io.FileNotFoundException" ] + firstLine = T.takeWhile (/= '\n') body + +isAccessDenied :: RemoteError -> Bool +isAccessDenied (RemoteError s _) = s == "org.apache.hadoop.security.AccessControlException"